Sep 11 16:17:23.466982 ip-10-0-132-119 systemd[1]: Starting Kubernetes Kubelet... Sep 11 16:17:23.837919 ip-10-0-132-119 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 16:17:23.837919 ip-10-0-132-119 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 11 16:17:23.837919 ip-10-0-132-119 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 16:17:23.837919 ip-10-0-132-119 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 11 16:17:23.837919 ip-10-0-132-119 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 11 16:17:23.839672 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.839586 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 11 16:17:23.842966 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842951 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 16:17:23.842966 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842966 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842970 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842974 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842977 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842980 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842982 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842985 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842988 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842991 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842994 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842996 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.842999 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843002 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843004 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843007 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843009 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843012 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843018 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843021 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 16:17:23.843026 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843023 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843027 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843030 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843032 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843035 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843038 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843040 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843043 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843046 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843048 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843051 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843053 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843056 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843060 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843063 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843066 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843069 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843071 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843074 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843076 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 16:17:23.843481 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843079 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843081 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843083 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843086 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843089 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843092 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843094 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843097 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843099 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843101 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843104 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843107 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843110 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843113 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843116 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843118 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843121 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843123 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843125 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843128 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 16:17:23.844114 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843130 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843133 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843135 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843138 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843140 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843144 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843146 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843149 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843151 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843153 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843156 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843158 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843160 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843163 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843165 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843167 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843171 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843175 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843178 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 16:17:23.844882 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843180 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843184 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843187 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843190 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843193 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843196 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843200 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843204 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843207 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843209 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843212 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843214 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843217 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843220 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843222 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843225 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843227 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843230 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843233 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.843236 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 16:17:23.845479 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845170 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845184 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845188 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845191 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845194 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845197 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845200 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845203 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845206 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845208 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845210 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845214 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845216 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845219 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845221 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845224 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845226 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845229 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845232 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845235 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 16:17:23.845988 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845238 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845241 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845246 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845249 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845252 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845254 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845257 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845260 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845262 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845265 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845267 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845270 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845272 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845275 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845278 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845281 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845283 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845286 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845288 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 16:17:23.846504 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845291 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845293 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845296 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845298 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845301 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845303 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845306 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845308 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845311 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845313 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845316 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845319 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845322 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845324 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845327 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845329 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845331 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845334 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845336 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 16:17:23.846983 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845339 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845342 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845345 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845347 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845350 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845356 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845360 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845364 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845367 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845369 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845372 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845374 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845377 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845379 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845383 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845386 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845389 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845392 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845394 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 16:17:23.847453 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845397 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845400 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845403 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845405 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845408 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845411 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845415 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845418 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845421 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845423 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845426 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845429 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845431 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845434 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845436 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845439 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845441 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845444 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845446 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845448 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 16:17:23.847944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845452 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845454 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845845 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845851 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845854 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845857 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845860 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845862 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845865 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845868 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845870 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845872 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845875 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845878 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845880 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845883 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845887 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845891 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845894 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 16:17:23.848422 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845898 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845901 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845904 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845906 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845909 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845912 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845915 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845918 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845920 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845922 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845925 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845929 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845931 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845935 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845937 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845939 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845942 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845944 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845947 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 16:17:23.848893 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845949 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845952 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845954 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845957 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845959 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845962 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845964 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845966 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845969 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845973 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845977 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845980 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845983 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845986 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845989 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845992 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845995 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.845998 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846001 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 16:17:23.849389 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846004 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846006 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846009 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846011 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846014 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846016 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846019 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846021 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846024 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846026 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846029 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846032 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846035 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846037 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846039 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846042 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846044 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846046 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846049 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846051 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846054 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 16:17:23.849855 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846056 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846058 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846061 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846066 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846068 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846070 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846073 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846076 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846078 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846081 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846083 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846086 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846088 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846090 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846093 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846096 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846098 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846101 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846104 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846106 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 16:17:23.850355 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846109 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846111 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846114 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846189 2553 flags.go:64] FLAG: --address="0.0.0.0" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846230 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846237 2553 flags.go:64] FLAG: --anonymous-auth="true" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846242 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846245 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846248 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846252 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846257 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846260 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846263 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846267 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846270 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846275 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846278 2553 flags.go:64] FLAG: --cgroup-root="" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846280 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846283 2553 flags.go:64] FLAG: --client-ca-file="" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846287 2553 flags.go:64] FLAG: --cloud-provider="external" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846290 2553 flags.go:64] FLAG: --cluster-dns="[]" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846294 2553 flags.go:64] FLAG: --cluster-domain="" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846297 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846300 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 11 16:17:23.850928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846303 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846306 2553 flags.go:64] FLAG: --container-log-max-files="5" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846314 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846317 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846320 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846323 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846326 2553 flags.go:64] FLAG: --contention-profiling="false" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846329 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846332 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846335 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846338 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846342 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846345 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846348 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846350 2553 flags.go:64] FLAG: --enable-load-reader="false" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846353 2553 flags.go:64] FLAG: --enable-server="true" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846356 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846360 2553 flags.go:64] FLAG: --event-burst="100" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846364 2553 flags.go:64] FLAG: --event-qps="50" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846366 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846369 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846372 2553 flags.go:64] FLAG: --eviction-hard="" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846376 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846380 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846383 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 11 16:17:23.851595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846386 2553 flags.go:64] FLAG: --eviction-soft="" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846388 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846391 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846394 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846397 2553 flags.go:64] FLAG: --experimental-mounter-path="" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846401 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846403 2553 flags.go:64] FLAG: --fail-swap-on="true" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846406 2553 flags.go:64] FLAG: --feature-gates="" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846410 2553 flags.go:64] FLAG: --file-check-frequency="20s" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846413 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846416 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846419 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846422 2553 flags.go:64] FLAG: --healthz-port="10248" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846427 2553 flags.go:64] FLAG: --help="false" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846430 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-132-119.ec2.internal" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846433 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846436 2553 flags.go:64] FLAG: --http-check-frequency="20s" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846439 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846442 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846445 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846448 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846450 2553 flags.go:64] FLAG: --image-service-endpoint="" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846453 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 11 16:17:23.852238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846456 2553 flags.go:64] FLAG: --kube-api-burst="100" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846459 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846462 2553 flags.go:64] FLAG: --kube-api-qps="50" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846464 2553 flags.go:64] FLAG: --kube-reserved="" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846467 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846470 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846472 2553 flags.go:64] FLAG: --kubelet-cgroups="" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846476 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846479 2553 flags.go:64] FLAG: --lock-file="" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846482 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846484 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846487 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846493 2553 flags.go:64] FLAG: --log-json-split-stream="false" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846496 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846499 2553 flags.go:64] FLAG: --log-text-split-stream="false" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846502 2553 flags.go:64] FLAG: --logging-format="text" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846505 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846508 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846511 2553 flags.go:64] FLAG: --manifest-url="" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846514 2553 flags.go:64] FLAG: --manifest-url-header="" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846519 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846522 2553 flags.go:64] FLAG: --max-open-files="1000000" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846527 2553 flags.go:64] FLAG: --max-pods="110" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846530 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846532 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 11 16:17:23.852841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846535 2553 flags.go:64] FLAG: --memory-manager-policy="None" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846538 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846541 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846544 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846547 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846554 2553 flags.go:64] FLAG: --node-status-max-images="50" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846557 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846560 2553 flags.go:64] FLAG: --oom-score-adj="-999" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846563 2553 flags.go:64] FLAG: --pod-cidr="" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846566 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846571 2553 flags.go:64] FLAG: --pod-manifest-path="" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846574 2553 flags.go:64] FLAG: --pod-max-pids="-1" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846578 2553 flags.go:64] FLAG: --pods-per-core="0" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846581 2553 flags.go:64] FLAG: --port="10250" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846583 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846588 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-057de041a40c705f4" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846591 2553 flags.go:64] FLAG: --qos-reserved="" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846593 2553 flags.go:64] FLAG: --read-only-port="10255" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846596 2553 flags.go:64] FLAG: --register-node="true" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846599 2553 flags.go:64] FLAG: --register-with-taints="" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846603 2553 flags.go:64] FLAG: --registry-burst="10" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846606 2553 flags.go:64] FLAG: --registry-qps="5" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846609 2553 flags.go:64] FLAG: --reserved-cpus="" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846612 2553 flags.go:64] FLAG: --reserved-memory="" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846615 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 11 16:17:23.853458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846618 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846621 2553 flags.go:64] FLAG: --rotate-certificates="false" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846624 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846626 2553 flags.go:64] FLAG: --runonce="false" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846630 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846633 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846636 2553 flags.go:64] FLAG: --seccomp-default="false" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846639 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846641 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846644 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846647 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846650 2553 flags.go:64] FLAG: --storage-driver-password="root" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846653 2553 flags.go:64] FLAG: --storage-driver-secure="false" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846656 2553 flags.go:64] FLAG: --storage-driver-table="stats" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846658 2553 flags.go:64] FLAG: --storage-driver-user="root" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846661 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846664 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846667 2553 flags.go:64] FLAG: --system-cgroups="" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846669 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846675 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846692 2553 flags.go:64] FLAG: --tls-cert-file="" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846695 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846700 2553 flags.go:64] FLAG: --tls-min-version="" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846703 2553 flags.go:64] FLAG: --tls-private-key-file="" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846705 2553 flags.go:64] FLAG: --topology-manager-policy="none" Sep 11 16:17:23.854079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846708 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846711 2553 flags.go:64] FLAG: --topology-manager-scope="container" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846714 2553 flags.go:64] FLAG: --v="2" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846718 2553 flags.go:64] FLAG: --version="false" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846722 2553 flags.go:64] FLAG: --vmodule="" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846726 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.846729 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846840 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846844 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846847 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846850 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846853 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846856 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846858 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846861 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846863 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846866 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846868 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846871 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846873 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846876 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 16:17:23.854728 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846879 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846881 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846884 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846886 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846889 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846891 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846894 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846896 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846900 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846903 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846906 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846908 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846910 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846913 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846915 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846918 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846921 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846924 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846926 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846929 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 16:17:23.855240 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846931 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846934 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846937 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846940 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846942 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846945 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846947 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846950 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846953 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846955 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846958 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846960 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846963 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846965 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846968 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846970 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846972 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846975 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846979 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846982 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 16:17:23.855745 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846986 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846989 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846992 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846995 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.846998 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847001 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847003 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847006 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847008 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847011 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847014 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847017 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847019 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847022 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847025 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847028 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847030 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847033 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847035 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847038 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 16:17:23.856243 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847040 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847043 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847046 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847048 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847050 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847053 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847055 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847058 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847060 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847063 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847065 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847068 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847071 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847074 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847076 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847080 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847084 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847088 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847091 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 16:17:23.856767 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847094 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847097 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847100 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847103 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847106 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.847109 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.847536 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.854827 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.854842 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.854861 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.854866 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854930 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854935 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854938 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 16:17:23.857229 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854942 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854945 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854948 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854951 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854953 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854956 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854959 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854961 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854964 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854966 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854969 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854972 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854974 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854977 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854979 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854982 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854984 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854988 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854992 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854995 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 16:17:23.857583 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.854997 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855001 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855003 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855006 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855008 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855011 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855014 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855017 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855020 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855022 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855025 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855027 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855030 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855032 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855035 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855038 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855040 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855043 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855046 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855049 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 16:17:23.858123 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855052 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855054 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855057 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855059 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855062 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855066 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855069 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855072 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855075 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855077 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855080 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855083 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855085 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855088 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855091 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855093 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855096 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855098 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855102 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 16:17:23.858602 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855105 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855107 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855110 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855112 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855115 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855117 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855119 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855122 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855126 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855129 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855132 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855135 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855137 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855140 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855143 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855146 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855149 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855151 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855154 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 16:17:23.859069 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855157 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855159 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855162 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855164 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855167 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855170 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855172 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855175 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855177 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855180 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855183 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855186 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855188 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855191 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855193 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855195 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855198 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 16:17:23.859521 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855200 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.855205 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855322 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855326 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855329 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855332 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855335 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855337 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855340 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855343 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855346 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855349 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855352 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855355 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 11 16:17:23.859949 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855358 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855360 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855363 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855365 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855368 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855371 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855373 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855375 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855378 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855380 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855383 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855395 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855398 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855401 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855403 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855406 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855408 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855411 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855413 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855416 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 11 16:17:23.860295 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855418 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855420 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855423 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855426 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855428 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855430 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855433 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855435 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855438 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855442 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855445 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855448 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855450 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855453 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855455 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855458 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855460 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855463 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855465 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855468 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 11 16:17:23.860789 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855470 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855472 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855475 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855477 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855480 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855483 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855485 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855488 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855490 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855493 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855496 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855498 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855500 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855503 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855505 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855508 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855510 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855513 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855516 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 11 16:17:23.861264 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855520 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855522 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855525 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855528 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855530 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855533 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855535 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855537 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855540 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855542 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855545 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855547 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855550 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855552 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855554 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855557 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855559 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855562 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855565 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855568 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 11 16:17:23.861832 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855570 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855572 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855575 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855577 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855581 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855584 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855588 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:23.855591 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.855596 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.855714 2553 server.go:962] "Client rotation is on, will bootstrap in background" Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.858017 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.858965 2553 server.go:1019] "Starting client certificate rotation" Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.859061 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 16:17:23.862310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.859099 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 11 16:17:23.862631 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.861574 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 16:17:23.862956 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.862930 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 11 16:17:23.874634 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.874615 2553 log.go:25] "Validated CRI v1 runtime API" Sep 11 16:17:23.878860 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.878842 2553 log.go:25] "Validated CRI v1 image API" Sep 11 16:17:23.879658 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.879642 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 11 16:17:23.883172 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.883153 2553 fs.go:135] Filesystem UUIDs: map[450302e9-4117-44f9-a03e-24109152bf7f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8746b3ba-258d-467f-99ee-24d1e0b9bfa0:/dev/nvme0n1p3] Sep 11 16:17:23.883234 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.883171 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 11 16:17:23.887921 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.887814 2553 manager.go:216] Machine: {Timestamp:2026-09-11 16:17:23.88676023 +0000 UTC m=+0.301857764 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099044 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2acc973878d875da19320c6084aa8d SystemUUID:ec2acc97-3878-d875-da19-320c6084aa8d BootID:572d03e3-577c-42ce-839d-75f8641cc3cd Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:87:fb:0d:c3:f3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:87:fb:0d:c3:f3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:6f:ce:33:d5:61 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 11 16:17:23.887921 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.887915 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 11 16:17:23.888034 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.887994 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 11 16:17:23.889014 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.888990 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 11 16:17:23.889151 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.889016 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-119.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 11 16:17:23.889818 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.889807 2553 topology_manager.go:138] "Creating topology manager with none policy" Sep 11 16:17:23.889818 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.889819 2553 container_manager_linux.go:309] "Creating device plugin manager" Sep 11 16:17:23.889900 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.889829 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 16:17:23.891158 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.891148 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 11 16:17:23.891735 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.891720 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 16:17:23.892233 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.892222 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 11 16:17:23.897155 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897143 2553 state_mem.go:36] "Initialized new in-memory state store" Sep 11 16:17:23.897270 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897261 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 11 16:17:23.897319 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897311 2553 kubelet.go:486] "Attempting to sync node with API server" Sep 11 16:17:23.897355 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897323 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 11 16:17:23.897355 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897337 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 11 16:17:23.897355 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897346 2553 kubelet.go:398] "Adding apiserver pod source" Sep 11 16:17:23.897355 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897353 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 11 16:17:23.897781 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897770 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 16:17:23.897832 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.897791 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 11 16:17:23.911274 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.911253 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 11 16:17:23.913986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.913972 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 11 16:17:23.914040 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.914003 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 11 16:17:23.915629 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915617 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 11 16:17:23.915705 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915664 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 11 16:17:23.915705 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915675 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 11 16:17:23.915705 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915700 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915715 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915724 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915745 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915754 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915764 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915772 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915784 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 11 16:17:23.915809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.915793 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 11 16:17:23.916618 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.916604 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 11 16:17:23.916618 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.916619 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 11 16:17:23.917456 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.917439 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 11 16:17:23.917506 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.917439 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-119.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 11 16:17:23.920757 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.920596 2553 server.go:1268] "Started kubelet" Sep 11 16:17:23.920839 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.920664 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 11 16:17:23.920839 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.920737 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 11 16:17:23.920948 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.920908 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 11 16:17:23.921289 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.921256 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-27d2w" Sep 11 16:17:23.921584 ip-10-0-132-119 systemd[1]: Started Kubernetes Kubelet. Sep 11 16:17:23.922416 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.922310 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 11 16:17:23.922880 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.922866 2553 server.go:310] "Adding debug handlers to kubelet server" Sep 11 16:17:23.925012 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.924996 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 11 16:17:23.927236 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.927217 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-119.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 11 16:17:23.929474 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.929449 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 11 16:17:23.929634 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.929615 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-27d2w" Sep 11 16:17:23.929820 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.926321 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-119.ec2.internal.18d4507ba31ac442 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-119.ec2.internal,UID:ip-10-0-132-119.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-119.ec2.internal,},FirstTimestamp:2026-09-11 16:17:23.92056941 +0000 UTC m=+0.335666943,LastTimestamp:2026-09-11 16:17:23.92056941 +0000 UTC m=+0.335666943,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-119.ec2.internal,}" Sep 11 16:17:23.929903 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.929831 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 11 16:17:23.930073 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.930056 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:23.930789 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.930708 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 11 16:17:23.930789 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.930717 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 11 16:17:23.931010 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.930993 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 11 16:17:23.931124 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931081 2553 reconstruct.go:97] "Volume reconstruction finished" Sep 11 16:17:23.931124 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931091 2553 reconciler.go:29] "Reconciler: start to sync state" Sep 11 16:17:23.931559 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931545 2553 factory.go:55] Registering systemd factory Sep 11 16:17:23.931559 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931561 2553 factory.go:223] Registration of the systemd container factory successfully Sep 11 16:17:23.931762 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931748 2553 factory.go:153] Registering CRI-O factory Sep 11 16:17:23.931812 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931765 2553 factory.go:223] Registration of the crio container factory successfully Sep 11 16:17:23.931812 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931808 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 11 16:17:23.931915 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931824 2553 factory.go:103] Registering Raw factory Sep 11 16:17:23.931915 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.931834 2553 manager.go:1196] Started watching for new ooms in manager Sep 11 16:17:23.932173 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.932157 2553 manager.go:349] Starting recovery of all containers Sep 11 16:17:23.937808 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.937787 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 16:17:23.940804 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.940788 2553 manager.go:354] Recovery completed Sep 11 16:17:23.942057 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.942037 2553 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-119.ec2.internal\" not found" node="ip-10-0-132-119.ec2.internal" Sep 11 16:17:23.946877 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.946866 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:23.950183 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950162 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:23.950246 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950189 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:23.950246 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950199 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:23.950792 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950781 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:23.950792 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950788 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 11 16:17:23.950890 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950797 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 11 16:17:23.950890 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.950814 2553 state_mem.go:36] "Initialized new in-memory state store" Sep 11 16:17:23.951861 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.951838 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:23.951901 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.951865 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:23.951901 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.951879 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:23.953124 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.953111 2553 policy_none.go:49] "None policy: Start" Sep 11 16:17:23.953173 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.953133 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 11 16:17:23.953173 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.953148 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 11 16:17:23.955357 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.955345 2553 policy_none.go:47] "Start" Sep 11 16:17:23.997275 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.997258 2553 manager.go:337] "Starting Device Plugin manager" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.997300 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.997313 2553 server.go:87] "Starting device plugin registration server" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.997915 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.997940 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.998304 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.998417 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:23.998426 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.998827 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 11 16:17:24.012189 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:23.998888 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.069508 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.069441 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 11 16:17:24.070730 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.070711 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 11 16:17:24.070787 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.070772 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 11 16:17:24.070820 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.070803 2553 kubelet.go:2448] "Starting kubelet main sync loop" Sep 11 16:17:24.070863 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.070849 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 11 16:17:24.073586 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.073567 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 16:17:24.099006 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.098986 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:24.099895 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.099879 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:24.099971 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.099909 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:24.099971 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.099919 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:24.099971 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.099941 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.109292 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.109273 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.109388 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.109295 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-119.ec2.internal\": node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.137010 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.136993 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.171077 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.171034 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal"] Sep 11 16:17:24.171192 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.171138 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:24.172954 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.172937 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:24.173058 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.172974 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:24.173058 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.172989 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:24.175156 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.175142 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:24.175294 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.175279 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.175341 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.175309 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:24.177977 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.177958 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:24.178079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.177960 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:24.178079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.177988 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:24.178079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.178002 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:24.178079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.178004 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:24.178079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.178018 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:24.180187 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.180171 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.180238 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.180208 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 11 16:17:24.180959 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.180945 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientMemory" Sep 11 16:17:24.181052 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.180974 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasNoDiskPressure" Sep 11 16:17:24.181052 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.180986 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeHasSufficientPID" Sep 11 16:17:24.199289 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.199269 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-119.ec2.internal\" not found" node="ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.202637 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.202623 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-119.ec2.internal\" not found" node="ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.232823 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.232790 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1af1352b21b56122291aa11ed6868de7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-119.ec2.internal\" (UID: \"1af1352b21b56122291aa11ed6868de7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.232823 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.232822 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/015e6701716bb3b67d61953ec545f312-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"015e6701716bb3b67d61953ec545f312\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.232982 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.232839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/015e6701716bb3b67d61953ec545f312-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"015e6701716bb3b67d61953ec545f312\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.237912 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.237894 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.333044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.332964 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1af1352b21b56122291aa11ed6868de7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-119.ec2.internal\" (UID: \"1af1352b21b56122291aa11ed6868de7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.333044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.332997 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/015e6701716bb3b67d61953ec545f312-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"015e6701716bb3b67d61953ec545f312\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.333044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.333014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/015e6701716bb3b67d61953ec545f312-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"015e6701716bb3b67d61953ec545f312\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.333206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.333093 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/015e6701716bb3b67d61953ec545f312-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"015e6701716bb3b67d61953ec545f312\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.333206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.333101 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/015e6701716bb3b67d61953ec545f312-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal\" (UID: \"015e6701716bb3b67d61953ec545f312\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.333206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.333094 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1af1352b21b56122291aa11ed6868de7-config\") pod \"kube-apiserver-proxy-ip-10-0-132-119.ec2.internal\" (UID: \"1af1352b21b56122291aa11ed6868de7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.338086 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.338059 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.439099 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.439070 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.502294 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.502257 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.505813 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.505797 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:24.539452 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.539422 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.640061 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.639979 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.740525 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.740493 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.841040 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.841018 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:24.859210 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.859184 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 11 16:17:24.859361 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.859337 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 16:17:24.859417 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.859353 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 11 16:17:24.933646 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.933563 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 16:12:23 +0000 UTC" deadline="2028-06-03 08:39:59.768278532 +0000 UTC" Sep 11 16:17:24.933646 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:24.933593 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15136h22m34.834688506s" Sep 11 16:17:24.941078 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:24.941056 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:25.041913 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:25.041879 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:25.102175 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.102150 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 16:17:25.142815 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:25.142784 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:25.183944 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:25.183873 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af1352b21b56122291aa11ed6868de7.slice/crio-3ee3292b591e90467b90724e206f3acaaff4fc1f633a8a61a003f76c49810693 WatchSource:0}: Error finding container 3ee3292b591e90467b90724e206f3acaaff4fc1f633a8a61a003f76c49810693: Status 404 returned error can't find the container with id 3ee3292b591e90467b90724e206f3acaaff4fc1f633a8a61a003f76c49810693 Sep 11 16:17:25.184334 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:25.184312 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015e6701716bb3b67d61953ec545f312.slice/crio-2da49297c88650966a0b162a43316b23260f0d9c53115d921d614b062a625210 WatchSource:0}: Error finding container 2da49297c88650966a0b162a43316b23260f0d9c53115d921d614b062a625210: Status 404 returned error can't find the container with id 2da49297c88650966a0b162a43316b23260f0d9c53115d921d614b062a625210 Sep 11 16:17:25.188900 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.188882 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 16:17:25.243806 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:25.243781 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-119.ec2.internal\" not found" Sep 11 16:17:25.269805 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.269776 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 16:17:25.330851 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.330820 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" Sep 11 16:17:25.344852 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.344827 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 16:17:25.345647 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.345633 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" Sep 11 16:17:25.363344 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.363329 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 11 16:17:25.899296 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.899262 2553 apiserver.go:52] "Watching apiserver" Sep 11 16:17:25.906550 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.906523 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 11 16:17:25.907721 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.907693 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bqpcw","openshift-image-registry/node-ca-62tgl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal","openshift-multus/network-metrics-daemon-5wxnc","openshift-network-operator/iptables-alerter-gx7gz","kube-system/konnectivity-agent-6b2pn","openshift-cluster-node-tuning-operator/tuned-86frm","openshift-multus/multus-additional-cni-plugins-6g24b","openshift-multus/multus-tzp4p","openshift-network-diagnostics/network-check-target-j7pkn","openshift-ovn-kubernetes/ovnkube-node-bdf94","kube-system/global-pull-secret-syncer-xrwjl","kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9"] Sep 11 16:17:25.910108 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.910079 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.912241 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.912222 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.915946 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.915929 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dd5v6\"" Sep 11 16:17:25.916256 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.916241 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.916598 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.916582 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.918310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.917941 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.918310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.918216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:25.918310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.918279 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.918510 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:25.918302 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:25.919508 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.919490 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:25.921430 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.921412 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 11 16:17:25.921735 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.921721 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xdk5b\"" Sep 11 16:17:25.921822 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.921794 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 11 16:17:25.922020 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922007 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.922196 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922182 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h5fcc\"" Sep 11 16:17:25.922261 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922202 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 11 16:17:25.922422 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922408 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.923211 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922704 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 11 16:17:25.923211 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922805 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 11 16:17:25.923211 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.922972 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.923388 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.923287 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.923496 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.923477 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 11 16:17:25.923700 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.923662 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plqzg\"" Sep 11 16:17:25.923784 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.923707 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.924520 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.924505 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:25.926759 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.926740 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 11 16:17:25.926759 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.926759 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.926870 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.926765 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 11 16:17:25.926901 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.926893 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pvbm2\"" Sep 11 16:17:25.929206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.928930 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.929206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.928985 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 11 16:17:25.929206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.929041 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 11 16:17:25.929206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.929100 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.929411 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.929219 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.929411 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.929344 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wxpqb\"" Sep 11 16:17:25.930421 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.930404 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 11 16:17:25.931154 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.931065 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:25.931857 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.931839 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xspvg\"" Sep 11 16:17:25.931941 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.931912 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 11 16:17:25.931941 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.931931 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 11 16:17:25.933057 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.933043 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.933535 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.933517 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.933593 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.933540 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 11 16:17:25.933714 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.933693 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fh5bs\"" Sep 11 16:17:25.934516 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.934500 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:25.934592 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:25.934572 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:25.936702 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.936670 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:25.938813 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.938796 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mj2s9\"" Sep 11 16:17:25.940180 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.940147 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 11 16:17:25.940390 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.940374 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 11 16:17:25.940781 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.940740 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:25.940871 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:25.940810 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:25.942257 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942235 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-kubernetes\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.942353 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942268 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-host\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.942353 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942290 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-run-netns\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942353 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942310 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-etc-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942353 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942334 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-device-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942361 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-cni-bin\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942377 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-system-cni-dir\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942400 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-slash\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942419 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942439 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-registration-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942485 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942524 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-var-lib-kubelet\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.942556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942553 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942584 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-systemd\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942619 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-cni-netd\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942662 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovn-node-metrics-cert\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942727 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a126d30a-b32c-4f2a-9bab-fc887339dddb-cni-binary-copy\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942786 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-cni-multus\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942819 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-tmp\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942842 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7kdh\" (UniqueName: \"kubernetes.io/projected/889874bb-c587-44d6-858f-207e25a501d6-kube-api-access-s7kdh\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942865 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-cnibin\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.942909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942886 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-cni-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942925 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovnkube-config\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.942963 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbdp4\" (UniqueName: \"kubernetes.io/projected/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-kube-api-access-sbdp4\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943011 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-kubelet\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943049 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-var-lib-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943088 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-run-ovn-kubernetes\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943128 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943160 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmmzc\" (UniqueName: \"kubernetes.io/projected/c3484cd6-fa9f-4e35-bfb7-15d16130f884-kube-api-access-gmmzc\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943191 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943201 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4gzm\" (UniqueName: \"kubernetes.io/projected/1a762e41-b42c-470f-bd77-4d790aed8532-kube-api-access-b4gzm\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:25.943220 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943223 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-system-cni-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943244 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-netns\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-sys-fs\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943309 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943333 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3484cd6-fa9f-4e35-bfb7-15d16130f884-iptables-alerter-script\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943355 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a762e41-b42c-470f-bd77-4d790aed8532-serviceca\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943377 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cnibin\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943417 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2vvl\" (UniqueName: \"kubernetes.io/projected/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-kube-api-access-h2vvl\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943460 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-socket-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943485 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phsdk\" (UniqueName: \"kubernetes.io/projected/4ac947c7-0e15-4a76-88b8-0004ce8933e7-kube-api-access-phsdk\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943520 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-socket-dir-parent\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943544 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-conf-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943566 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysctl-conf\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943595 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-lib-modules\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943628 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-tuned\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943648 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-systemd\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-log-socket\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943717 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-kubelet\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943738 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-hostroot\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943759 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-modprobe-d\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943784 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovnkube-script-lib\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943833 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-env-overrides\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a762e41-b42c-470f-bd77-4d790aed8532-host\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943882 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943906 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943940 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-node-log\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.943986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943968 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3484cd6-fa9f-4e35-bfb7-15d16130f884-host-slash\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.943988 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-k8s-cni-cncf-io\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944015 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-etc-kubernetes\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944044 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944073 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysctl-d\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944102 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-run\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944126 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-ovn\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944146 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-cni-bin\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944169 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2a9f964-4217-4878-8c24-b086a1202c5d-konnectivity-ca\") pod \"konnectivity-agent-6b2pn\" (UID: \"a2a9f964-4217-4878-8c24-b086a1202c5d\") " pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944194 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-daemon-config\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944238 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdgrv\" (UniqueName: \"kubernetes.io/projected/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-kube-api-access-wdgrv\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944262 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysconfig\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944281 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-sys\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944301 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-systemd-units\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944322 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944343 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2a9f964-4217-4878-8c24-b086a1202c5d-agent-certs\") pod \"konnectivity-agent-6b2pn\" (UID: \"a2a9f964-4217-4878-8c24-b086a1202c5d\") " pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:25.944499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-multus-certs\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:25.945062 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:25.944386 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgwrv\" (UniqueName: \"kubernetes.io/projected/a126d30a-b32c-4f2a-9bab-fc887339dddb-kube-api-access-cgwrv\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.013308 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.013279 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wztkh" Sep 11 16:17:26.024169 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.024140 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wztkh" Sep 11 16:17:26.032329 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.032310 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 11 16:17:26.045560 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045532 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-etc-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045571 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-device-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-cni-bin\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045627 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-system-cni-dir\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045640 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-etc-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045648 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-cni-bin\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045650 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-slash\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045696 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-device-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.045719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045704 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045722 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-system-cni-dir\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045730 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-registration-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045722 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-slash\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045752 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045778 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0475e5f9-fcb0-4165-b925-ca7901a3b53d-hosts-file\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045782 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045804 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-registration-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045821 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-dbus\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045847 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-var-lib-kubelet\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045870 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045871 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045906 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-systemd\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045911 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-var-lib-kubelet\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045931 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-cni-netd\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045955 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-systemd\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovn-node-metrics-cert\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045981 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.045987 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a126d30a-b32c-4f2a-9bab-fc887339dddb-cni-binary-copy\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046026 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-cni-multus\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046033 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-cni-netd\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046049 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-tmp\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046074 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-cni-multus\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046087 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7kdh\" (UniqueName: \"kubernetes.io/projected/889874bb-c587-44d6-858f-207e25a501d6-kube-api-access-s7kdh\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046107 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-cnibin\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046127 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-cni-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovnkube-config\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046185 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbdp4\" (UniqueName: \"kubernetes.io/projected/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-kube-api-access-sbdp4\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046207 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-kubelet\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046229 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-var-lib-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046250 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-run-ovn-kubernetes\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046260 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-cni-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046273 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046309 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2ktb\" (UniqueName: \"kubernetes.io/projected/0475e5f9-fcb0-4165-b925-ca7901a3b53d-kube-api-access-l2ktb\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.046844 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046335 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-kubelet-config\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmmzc\" (UniqueName: \"kubernetes.io/projected/c3484cd6-fa9f-4e35-bfb7-15d16130f884-kube-api-access-gmmzc\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046383 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4gzm\" (UniqueName: \"kubernetes.io/projected/1a762e41-b42c-470f-bd77-4d790aed8532-kube-api-access-b4gzm\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046408 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-system-cni-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046430 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-netns\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046463 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-sys-fs\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046466 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a126d30a-b32c-4f2a-9bab-fc887339dddb-cni-binary-copy\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046505 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-cnibin\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046544 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-kubelet\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046651 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046722 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-system-cni-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046790 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-sys-fs\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046830 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046865 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-var-lib-openvswitch\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046870 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046837 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-netns\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046887 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-run-ovn-kubernetes\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046908 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3484cd6-fa9f-4e35-bfb7-15d16130f884-iptables-alerter-script\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.047572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046933 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a762e41-b42c-470f-bd77-4d790aed8532-serviceca\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046955 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cnibin\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.046962 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.046979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vvl\" (UniqueName: \"kubernetes.io/projected/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-kube-api-access-h2vvl\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047002 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-socket-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.047028 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:26.547009789 +0000 UTC m=+2.962107333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047057 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phsdk\" (UniqueName: \"kubernetes.io/projected/4ac947c7-0e15-4a76-88b8-0004ce8933e7-kube-api-access-phsdk\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047084 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-socket-dir-parent\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047103 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-socket-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047108 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-conf-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047130 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysctl-conf\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047151 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-lib-modules\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047168 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-tuned\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047185 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-systemd\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047206 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-log-socket\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047227 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-kubelet\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047247 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-hostroot\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.048382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047270 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-modprobe-d\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047294 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovnkube-script-lib\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047264 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovnkube-config\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047321 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0475e5f9-fcb0-4165-b925-ca7901a3b53d-tmp-dir\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047344 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-env-overrides\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047360 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a762e41-b42c-470f-bd77-4d790aed8532-serviceca\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047371 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a762e41-b42c-470f-bd77-4d790aed8532-host\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047364 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-socket-dir-parent\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047417 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cnibin\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047439 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-lib-modules\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047449 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047479 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-conf-dir\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047487 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047516 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-node-log\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047542 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047441 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c3484cd6-fa9f-4e35-bfb7-15d16130f884-iptables-alerter-script\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047583 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysctl-conf\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049177 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047583 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047635 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-node-log\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047660 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3484cd6-fa9f-4e35-bfb7-15d16130f884-host-slash\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047666 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-modprobe-d\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047707 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-k8s-cni-cncf-io\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047722 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-systemd\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047732 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-etc-kubernetes\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047755 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-log-socket\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047755 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047784 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysctl-d\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047811 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-run\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047835 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-ovn\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047856 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-cni-bin\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047879 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2a9f964-4217-4878-8c24-b086a1202c5d-konnectivity-ca\") pod \"konnectivity-agent-6b2pn\" (UID: \"a2a9f964-4217-4878-8c24-b086a1202c5d\") " pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047902 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-daemon-config\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047921 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdgrv\" (UniqueName: \"kubernetes.io/projected/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-kube-api-access-wdgrv\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047948 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysconfig\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047968 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-sys\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.049764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.047989 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-systemd-units\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048011 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048035 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048060 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2a9f964-4217-4878-8c24-b086a1202c5d-agent-certs\") pod \"konnectivity-agent-6b2pn\" (UID: \"a2a9f964-4217-4878-8c24-b086a1202c5d\") " pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048087 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-multus-certs\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048110 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgwrv\" (UniqueName: \"kubernetes.io/projected/a126d30a-b32c-4f2a-9bab-fc887339dddb-kube-api-access-cgwrv\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048133 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-kubernetes\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048153 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-host\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048174 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-run-netns\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048206 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048215 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048238 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovnkube-script-lib\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048241 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-run-netns\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048268 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-var-lib-kubelet\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048270 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-etc-kubernetes\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048303 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-k8s-cni-cncf-io\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048306 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3484cd6-fa9f-4e35-bfb7-15d16130f884-host-slash\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.050374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048321 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/889874bb-c587-44d6-858f-207e25a501d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048315 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a762e41-b42c-470f-bd77-4d790aed8532-host\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048348 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysconfig\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048353 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-run-ovn\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048368 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-systemd-units\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048382 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-host-cni-bin\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048409 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-sys\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048485 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-kubernetes\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048516 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-host-run-multus-certs\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048695 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-run\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048708 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-host\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048787 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a2a9f964-4217-4878-8c24-b086a1202c5d-konnectivity-ca\") pod \"konnectivity-agent-6b2pn\" (UID: \"a2a9f964-4217-4878-8c24-b086a1202c5d\") " pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048788 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-sysctl-d\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048807 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a126d30a-b32c-4f2a-9bab-fc887339dddb-multus-daemon-config\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.048815 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a126d30a-b32c-4f2a-9bab-fc887339dddb-hostroot\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.050917 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.049069 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-env-overrides\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.052076 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.051917 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-etc-tuned\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.052178 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.051937 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-tmp\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.052178 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.052126 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a2a9f964-4217-4878-8c24-b086a1202c5d-agent-certs\") pod \"konnectivity-agent-6b2pn\" (UID: \"a2a9f964-4217-4878-8c24-b086a1202c5d\") " pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:26.052328 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.052314 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-ovn-node-metrics-cert\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.058253 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.058173 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7kdh\" (UniqueName: \"kubernetes.io/projected/889874bb-c587-44d6-858f-207e25a501d6-kube-api-access-s7kdh\") pod \"aws-ebs-csi-driver-node-lknf9\" (UID: \"889874bb-c587-44d6-858f-207e25a501d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.059817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.059615 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbdp4\" (UniqueName: \"kubernetes.io/projected/c9efec09-dc0f-41b8-8dfd-692abdc6d11c-kube-api-access-sbdp4\") pod \"tuned-86frm\" (UID: \"c9efec09-dc0f-41b8-8dfd-692abdc6d11c\") " pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.060112 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.060094 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgwrv\" (UniqueName: \"kubernetes.io/projected/a126d30a-b32c-4f2a-9bab-fc887339dddb-kube-api-access-cgwrv\") pod \"multus-tzp4p\" (UID: \"a126d30a-b32c-4f2a-9bab-fc887339dddb\") " pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.060211 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.060140 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phsdk\" (UniqueName: \"kubernetes.io/projected/4ac947c7-0e15-4a76-88b8-0004ce8933e7-kube-api-access-phsdk\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:26.060655 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.060614 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vvl\" (UniqueName: \"kubernetes.io/projected/6cb0b991-2dce-4a62-b6a6-30e8dd32023d-kube-api-access-h2vvl\") pod \"ovnkube-node-bdf94\" (UID: \"6cb0b991-2dce-4a62-b6a6-30e8dd32023d\") " pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.060949 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.060928 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4gzm\" (UniqueName: \"kubernetes.io/projected/1a762e41-b42c-470f-bd77-4d790aed8532-kube-api-access-b4gzm\") pod \"node-ca-62tgl\" (UID: \"1a762e41-b42c-470f-bd77-4d790aed8532\") " pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.061020 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.060928 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdgrv\" (UniqueName: \"kubernetes.io/projected/a94516ff-ab02-4f35-82d4-6ac996fc5cb2-kube-api-access-wdgrv\") pod \"multus-additional-cni-plugins-6g24b\" (UID: \"a94516ff-ab02-4f35-82d4-6ac996fc5cb2\") " pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.061901 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.061880 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmmzc\" (UniqueName: \"kubernetes.io/projected/c3484cd6-fa9f-4e35-bfb7-15d16130f884-kube-api-access-gmmzc\") pod \"iptables-alerter-gx7gz\" (UID: \"c3484cd6-fa9f-4e35-bfb7-15d16130f884\") " pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.075357 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.075315 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" event={"ID":"015e6701716bb3b67d61953ec545f312","Type":"ContainerStarted","Data":"2da49297c88650966a0b162a43316b23260f0d9c53115d921d614b062a625210"} Sep 11 16:17:26.076476 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.076452 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" event={"ID":"1af1352b21b56122291aa11ed6868de7","Type":"ContainerStarted","Data":"3ee3292b591e90467b90724e206f3acaaff4fc1f633a8a61a003f76c49810693"} Sep 11 16:17:26.117785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.117757 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 16:17:26.148622 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148586 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2ktb\" (UniqueName: \"kubernetes.io/projected/0475e5f9-fcb0-4165-b925-ca7901a3b53d-kube-api-access-l2ktb\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.148622 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148623 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-kubelet-config\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148666 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0475e5f9-fcb0-4165-b925-ca7901a3b53d-tmp-dir\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148744 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148748 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-kubelet-config\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148776 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0475e5f9-fcb0-4165-b925-ca7901a3b53d-hosts-file\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148813 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-dbus\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.148863 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148824 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0475e5f9-fcb0-4165-b925-ca7901a3b53d-hosts-file\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.149069 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.148951 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:26.149069 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.148991 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:26.648978228 +0000 UTC m=+3.064075747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:26.149069 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.148996 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-dbus\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.149454 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.149411 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0475e5f9-fcb0-4165-b925-ca7901a3b53d-tmp-dir\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.157494 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.157473 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:26.157494 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.157496 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:26.157708 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.157507 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:26.157708 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.157557 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:26.65754271 +0000 UTC m=+3.072640248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:26.160407 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.160390 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2ktb\" (UniqueName: \"kubernetes.io/projected/0475e5f9-fcb0-4165-b925-ca7901a3b53d-kube-api-access-l2ktb\") pod \"node-resolver-bqpcw\" (UID: \"0475e5f9-fcb0-4165-b925-ca7901a3b53d\") " pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.226211 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.226175 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-86frm" Sep 11 16:17:26.234581 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.234554 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:26.244062 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.244038 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" Sep 11 16:17:26.248642 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.248617 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gx7gz" Sep 11 16:17:26.255210 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.255183 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:26.263414 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.263393 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tzp4p" Sep 11 16:17:26.270766 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.270746 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6g24b" Sep 11 16:17:26.278262 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.278239 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-62tgl" Sep 11 16:17:26.282749 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.282728 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bqpcw" Sep 11 16:17:26.367862 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.367834 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 11 16:17:26.552664 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.552585 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:26.552834 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.552750 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:26.552834 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.552824 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.552805966 +0000 UTC m=+3.967903498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:26.653438 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.653395 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:26.653586 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.653525 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:26.653633 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.653590 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.653569069 +0000 UTC m=+4.068666589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:26.753738 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:26.753699 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:26.753913 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.753811 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:26.753913 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.753825 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:26.753913 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.753836 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:26.753913 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:26.753893 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.753876514 +0000 UTC m=+4.168974042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:26.940090 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.940051 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94516ff_ab02_4f35_82d4_6ac996fc5cb2.slice/crio-d29f96e2e65c2a27111b640a7dd5a6da147c586491b93474c77e0be9c9a60a79 WatchSource:0}: Error finding container d29f96e2e65c2a27111b640a7dd5a6da147c586491b93474c77e0be9c9a60a79: Status 404 returned error can't find the container with id d29f96e2e65c2a27111b640a7dd5a6da147c586491b93474c77e0be9c9a60a79 Sep 11 16:17:26.941655 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.941636 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a9f964_4217_4878_8c24_b086a1202c5d.slice/crio-8418e0f5d6bb13125fc89f81fd6d587de711fefde04f22bb2dc0127fd8511bac WatchSource:0}: Error finding container 8418e0f5d6bb13125fc89f81fd6d587de711fefde04f22bb2dc0127fd8511bac: Status 404 returned error can't find the container with id 8418e0f5d6bb13125fc89f81fd6d587de711fefde04f22bb2dc0127fd8511bac Sep 11 16:17:26.942356 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.942336 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0475e5f9_fcb0_4165_b925_ca7901a3b53d.slice/crio-f0f02e37f7e5a6c26786a40a4c2d64a240b097bb99c11d4fa0bb14d8edda05e5 WatchSource:0}: Error finding container f0f02e37f7e5a6c26786a40a4c2d64a240b097bb99c11d4fa0bb14d8edda05e5: Status 404 returned error can't find the container with id f0f02e37f7e5a6c26786a40a4c2d64a240b097bb99c11d4fa0bb14d8edda05e5 Sep 11 16:17:26.943736 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.943713 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda126d30a_b32c_4f2a_9bab_fc887339dddb.slice/crio-18a3f726abf3629af0cd18472f1909603f0b7adbb69bd1cdd921468f264e5b2e WatchSource:0}: Error finding container 18a3f726abf3629af0cd18472f1909603f0b7adbb69bd1cdd921468f264e5b2e: Status 404 returned error can't find the container with id 18a3f726abf3629af0cd18472f1909603f0b7adbb69bd1cdd921468f264e5b2e Sep 11 16:17:26.946380 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.946358 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9efec09_dc0f_41b8_8dfd_692abdc6d11c.slice/crio-84905264310da49e5bdb4aaa7a3045786cb8096fa031a7378905c57633ea315b WatchSource:0}: Error finding container 84905264310da49e5bdb4aaa7a3045786cb8096fa031a7378905c57633ea315b: Status 404 returned error can't find the container with id 84905264310da49e5bdb4aaa7a3045786cb8096fa031a7378905c57633ea315b Sep 11 16:17:26.948049 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.947924 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb0b991_2dce_4a62_b6a6_30e8dd32023d.slice/crio-e2b57553b6f4468e21aa9e5f574bd67920390659f2ba523824481cc03c210fcd WatchSource:0}: Error finding container e2b57553b6f4468e21aa9e5f574bd67920390659f2ba523824481cc03c210fcd: Status 404 returned error can't find the container with id e2b57553b6f4468e21aa9e5f574bd67920390659f2ba523824481cc03c210fcd Sep 11 16:17:26.949565 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.949513 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3484cd6_fa9f_4e35_bfb7_15d16130f884.slice/crio-d5d56ec09c3dd3fa2bfc7b9dc657d2cde67e50e56f77c42c66282cc01449f0b4 WatchSource:0}: Error finding container d5d56ec09c3dd3fa2bfc7b9dc657d2cde67e50e56f77c42c66282cc01449f0b4: Status 404 returned error can't find the container with id d5d56ec09c3dd3fa2bfc7b9dc657d2cde67e50e56f77c42c66282cc01449f0b4 Sep 11 16:17:26.950552 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.950524 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889874bb_c587_44d6_858f_207e25a501d6.slice/crio-269ff3cc3b50b8e98f0c0e3e1a5a87efc6c3a69f912243dce5180f1799bf07e8 WatchSource:0}: Error finding container 269ff3cc3b50b8e98f0c0e3e1a5a87efc6c3a69f912243dce5180f1799bf07e8: Status 404 returned error can't find the container with id 269ff3cc3b50b8e98f0c0e3e1a5a87efc6c3a69f912243dce5180f1799bf07e8 Sep 11 16:17:26.951928 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:26.951907 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a762e41_b42c_470f_bd77_4d790aed8532.slice/crio-7706ac1f3b5c0a4b7f250f87377532b590a7edc719938a12aa47cc28c3286d7e WatchSource:0}: Error finding container 7706ac1f3b5c0a4b7f250f87377532b590a7edc719938a12aa47cc28c3286d7e: Status 404 returned error can't find the container with id 7706ac1f3b5c0a4b7f250f87377532b590a7edc719938a12aa47cc28c3286d7e Sep 11 16:17:27.027121 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.027094 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:12:25 +0000 UTC" deadline="2028-03-03 04:16:59.864051781 +0000 UTC" Sep 11 16:17:27.027121 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.027118 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12923h59m32.83693643s" Sep 11 16:17:27.071630 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.071606 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:27.071630 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.071630 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:27.071797 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.071736 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:27.071865 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.071848 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:27.079004 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.078976 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-62tgl" event={"ID":"1a762e41-b42c-470f-bd77-4d790aed8532","Type":"ContainerStarted","Data":"7706ac1f3b5c0a4b7f250f87377532b590a7edc719938a12aa47cc28c3286d7e"} Sep 11 16:17:27.080134 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.080099 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" event={"ID":"889874bb-c587-44d6-858f-207e25a501d6","Type":"ContainerStarted","Data":"269ff3cc3b50b8e98f0c0e3e1a5a87efc6c3a69f912243dce5180f1799bf07e8"} Sep 11 16:17:27.082731 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.082712 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tzp4p" event={"ID":"a126d30a-b32c-4f2a-9bab-fc887339dddb","Type":"ContainerStarted","Data":"18a3f726abf3629af0cd18472f1909603f0b7adbb69bd1cdd921468f264e5b2e"} Sep 11 16:17:27.084178 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.084157 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6b2pn" event={"ID":"a2a9f964-4217-4878-8c24-b086a1202c5d","Type":"ContainerStarted","Data":"8418e0f5d6bb13125fc89f81fd6d587de711fefde04f22bb2dc0127fd8511bac"} Sep 11 16:17:27.086209 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.086188 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" event={"ID":"1af1352b21b56122291aa11ed6868de7","Type":"ContainerStarted","Data":"a08eb233323728119f79c3a039376c7a52ddc50e8f8a6586026fe5efbc7e8b34"} Sep 11 16:17:27.087271 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.087248 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gx7gz" event={"ID":"c3484cd6-fa9f-4e35-bfb7-15d16130f884","Type":"ContainerStarted","Data":"d5d56ec09c3dd3fa2bfc7b9dc657d2cde67e50e56f77c42c66282cc01449f0b4"} Sep 11 16:17:27.088246 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.088223 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"e2b57553b6f4468e21aa9e5f574bd67920390659f2ba523824481cc03c210fcd"} Sep 11 16:17:27.089268 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.089233 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-86frm" event={"ID":"c9efec09-dc0f-41b8-8dfd-692abdc6d11c","Type":"ContainerStarted","Data":"84905264310da49e5bdb4aaa7a3045786cb8096fa031a7378905c57633ea315b"} Sep 11 16:17:27.090471 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.090450 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bqpcw" event={"ID":"0475e5f9-fcb0-4165-b925-ca7901a3b53d","Type":"ContainerStarted","Data":"f0f02e37f7e5a6c26786a40a4c2d64a240b097bb99c11d4fa0bb14d8edda05e5"} Sep 11 16:17:27.091456 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.091439 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerStarted","Data":"d29f96e2e65c2a27111b640a7dd5a6da147c586491b93474c77e0be9c9a60a79"} Sep 11 16:17:27.559540 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.559461 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:27.559770 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.559606 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:27.559770 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.559691 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:29.559656704 +0000 UTC m=+5.974754239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:27.660046 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.660004 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:27.660216 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.660197 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:27.660267 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.660255 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:29.660238507 +0000 UTC m=+6.075336044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:27.761345 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:27.761310 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:27.761511 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.761486 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:27.761511 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.761504 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:27.761623 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.761516 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:27.761623 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:27.761567 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:29.76155175 +0000 UTC m=+6.176649275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:28.027907 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:28.027814 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:12:25 +0000 UTC" deadline="2028-03-19 06:51:47.727694277 +0000 UTC" Sep 11 16:17:28.027907 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:28.027848 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13310h34m19.699850475s" Sep 11 16:17:28.073641 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:28.073613 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:28.073811 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:28.073751 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:28.113736 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:28.112488 2553 generic.go:356] "Generic (PLEG): container finished" podID="015e6701716bb3b67d61953ec545f312" containerID="f16b0e4d5979524cf497043179305a51374bf285bcd3ab66bd1a204e4c23e565" exitCode=0 Sep 11 16:17:28.113736 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:28.113451 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" event={"ID":"015e6701716bb3b67d61953ec545f312","Type":"ContainerDied","Data":"f16b0e4d5979524cf497043179305a51374bf285bcd3ab66bd1a204e4c23e565"} Sep 11 16:17:28.129350 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:28.129302 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-119.ec2.internal" podStartSLOduration=3.129286698 podStartE2EDuration="3.129286698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:27.106822418 +0000 UTC m=+3.521919958" watchObservedRunningTime="2026-09-11 16:17:28.129286698 +0000 UTC m=+4.544384240" Sep 11 16:17:29.072944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:29.072154 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:29.072944 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.072278 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:29.072944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:29.072648 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:29.072944 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.072799 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:29.128611 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:29.127857 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" event={"ID":"015e6701716bb3b67d61953ec545f312","Type":"ContainerStarted","Data":"1cf4916d13d9f2c80b6eed43562c8be725ae4444bbddd0ed0280e5272e8f8d0e"} Sep 11 16:17:29.576964 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:29.576337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:29.576964 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.576525 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:29.576964 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.576586 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:33.576568093 +0000 UTC m=+9.991665631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:29.676976 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:29.676939 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:29.677150 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.677086 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:29.677206 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.677196 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:33.677176733 +0000 UTC m=+10.092274258 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:29.777882 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:29.777835 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:29.778067 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.778039 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:29.778067 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.778061 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:29.778214 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.778074 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:29.778214 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:29.778135 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:33.778118279 +0000 UTC m=+10.193215823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:30.072889 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:30.072807 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:30.073050 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:30.072941 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:31.071283 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:31.071245 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:31.071471 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:31.071386 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:31.071791 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:31.071771 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:31.071926 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:31.071854 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:32.071537 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:32.071501 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:32.072012 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:32.071613 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:33.071855 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:33.071821 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:33.072278 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.071965 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:33.072278 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:33.072004 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:33.072278 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.072085 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:33.621654 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:33.621608 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:33.621846 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.621832 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:33.621914 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.621907 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:41.621879415 +0000 UTC m=+18.036976949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:33.722470 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:33.722435 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:33.722675 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.722567 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:33.722675 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.722626 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:41.722609529 +0000 UTC m=+18.137707051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:33.823341 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:33.823309 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:33.823515 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.823494 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:33.823589 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.823523 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:33.823589 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.823536 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:33.823671 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:33.823591 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:41.823574249 +0000 UTC m=+18.238671776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:34.072299 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:34.072002 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:34.072729 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:34.072347 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:35.072451 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:35.072398 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:35.072451 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:35.072438 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:35.072974 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:35.072533 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:35.072974 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:35.072603 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:36.071439 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:36.071285 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:36.071439 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:36.071420 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:37.071151 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:37.071119 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:37.071570 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:37.071119 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:37.071570 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:37.071259 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:37.071570 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:37.071310 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:38.071260 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:38.071217 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:38.071760 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:38.071349 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:39.071255 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:39.071222 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:39.071440 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:39.071270 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:39.071440 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:39.071342 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:39.071831 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:39.071496 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:40.076516 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:40.076487 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:40.076967 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:40.076587 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:41.071829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:41.071795 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:41.072096 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:41.071847 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:41.072096 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.071934 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:41.072096 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.072078 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:41.678931 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:41.678895 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:41.679483 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.679079 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:41.679483 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.679154 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:57.679133381 +0000 UTC m=+34.094230917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:41.779938 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:41.779899 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:41.780118 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.780056 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:41.780181 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.780129 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:57.780110948 +0000 UTC m=+34.195208469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:41.880666 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:41.880625 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:41.880904 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.880797 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:41.880904 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.880819 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:41.880904 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.880833 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:41.880904 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:41.880884 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:57.880870662 +0000 UTC m=+34.295968188 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:42.071580 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:42.071489 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:42.071766 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:42.071607 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:43.071624 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:43.071591 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:43.072071 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:43.071602 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:43.072071 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:43.071730 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:43.072071 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:43.071818 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:44.072503 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:44.072468 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:44.072982 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:44.072563 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:45.071005 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:45.070983 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:45.071108 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:45.071088 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:45.071146 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:45.071120 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:45.071191 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:45.071179 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:46.071567 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.071409 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:46.072181 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:46.071658 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:46.144089 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.144065 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 11 16:17:46.167162 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.167122 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-62tgl" event={"ID":"1a762e41-b42c-470f-bd77-4d790aed8532","Type":"ContainerStarted","Data":"d35727e953231e87780d0afd2447d22bf3aa5dfd9f220f3e89381416bfe90638"} Sep 11 16:17:46.168828 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.168802 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" event={"ID":"889874bb-c587-44d6-858f-207e25a501d6","Type":"ContainerStarted","Data":"b9e3db99435caf0de86d87496c68d727939bc079bf36f8e8a690a9b24bc843a0"} Sep 11 16:17:46.168955 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.168847 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" event={"ID":"889874bb-c587-44d6-858f-207e25a501d6","Type":"ContainerStarted","Data":"8b5a19c8c1d79fc562a1ad75844fe1b577393c244449761ea53061cdc4a344c5"} Sep 11 16:17:46.170245 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.170221 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tzp4p" event={"ID":"a126d30a-b32c-4f2a-9bab-fc887339dddb","Type":"ContainerStarted","Data":"e9efc0df91a8c18676bd86bba2dd3ecb46ae215c66732d97a66c7fdfe94cf592"} Sep 11 16:17:46.171670 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.171644 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6b2pn" event={"ID":"a2a9f964-4217-4878-8c24-b086a1202c5d","Type":"ContainerStarted","Data":"8ea8c87ea28f5cb7c6795d0dc45ad5d7576390c756b6ad84e4ee5395ed0e9c3a"} Sep 11 16:17:46.174717 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.174665 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"48fc96be1a255ca3c83575f4a38cbadb683de05cd397fc8900940b3176318e3e"} Sep 11 16:17:46.174817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.174764 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"102cc9b82ada0f14e6a267cf4ec824f2d399e4c5928cfdf0e639c8de5cc2bc67"} Sep 11 16:17:46.174817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.174805 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"bb5c2cd5d6b6937a32efa8d8c19c93fc9508ca507cfb5c5de539ab393bb1de70"} Sep 11 16:17:46.174923 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.174835 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"87500716f7e7788726a1df5c86fb99fd2efac04dc89c741e35a6d9cec012873c"} Sep 11 16:17:46.174923 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.174875 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"a0c7e535cde44330dcb84fbcf07a86ec3e8052df849df33999d289d9d3532885"} Sep 11 16:17:46.174923 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.174904 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"bfc4a58454a5ffb741739fabfb82d8896a5e793b66c3af4532b866574348fb15"} Sep 11 16:17:46.176113 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.176091 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-86frm" event={"ID":"c9efec09-dc0f-41b8-8dfd-692abdc6d11c","Type":"ContainerStarted","Data":"71114ec0f1cce8b068ef1f8a110f5d4609b80efc8e7029bb529c9454e95f78bb"} Sep 11 16:17:46.177422 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.177403 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bqpcw" event={"ID":"0475e5f9-fcb0-4165-b925-ca7901a3b53d","Type":"ContainerStarted","Data":"438d1e6d92565adf646cca93fbd1678ee0ef42702308ea4b1b6f1edb6a4202dd"} Sep 11 16:17:46.178817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.178796 2553 generic.go:356] "Generic (PLEG): container finished" podID="a94516ff-ab02-4f35-82d4-6ac996fc5cb2" containerID="a67396d8ec31d98921b73b3d5687a54b9947550c8997166a2c143a90b515821b" exitCode=0 Sep 11 16:17:46.178898 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.178824 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerDied","Data":"a67396d8ec31d98921b73b3d5687a54b9947550c8997166a2c143a90b515821b"} Sep 11 16:17:46.188097 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.188057 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-119.ec2.internal" podStartSLOduration=21.18804432 podStartE2EDuration="21.18804432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:29.146632804 +0000 UTC m=+5.561730348" watchObservedRunningTime="2026-09-11 16:17:46.18804432 +0000 UTC m=+22.603141864" Sep 11 16:17:46.205881 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.205846 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-62tgl" podStartSLOduration=5.122099824 podStartE2EDuration="23.205833061s" totalImagesPullingTime="18.083733237s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.188861302 +0000 UTC m=+22.603958844" watchObservedRunningTime="2026-09-11 16:17:46.205833061 +0000 UTC m=+22.620930604" Sep 11 16:17:46.231973 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.231934 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6b2pn" podStartSLOduration=12.758784594 podStartE2EDuration="23.231923067s" totalImagesPullingTime="10.473138473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.207017631 +0000 UTC m=+22.622115175" watchObservedRunningTime="2026-09-11 16:17:46.231923067 +0000 UTC m=+22.647020609" Sep 11 16:17:46.251937 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.251902 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bqpcw" podStartSLOduration=5.156551916 podStartE2EDuration="23.251889029s" totalImagesPullingTime="18.095337113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.251251411 +0000 UTC m=+22.666348953" watchObservedRunningTime="2026-09-11 16:17:46.251889029 +0000 UTC m=+22.666986572" Sep 11 16:17:46.273046 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:46.273013 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-86frm" podStartSLOduration=5.17923571 podStartE2EDuration="23.273001498s" totalImagesPullingTime="18.093765788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.272502113 +0000 UTC m=+22.687599656" watchObservedRunningTime="2026-09-11 16:17:46.273001498 +0000 UTC m=+22.688099047" Sep 11 16:17:47.015338 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.015052 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T16:17:46.144085049Z","UUID":"db4b2dfc-4398-4ce8-b2a7-abf9d5ea96f3","Handler":null,"Name":"","Endpoint":""} Sep 11 16:17:47.017472 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.017447 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 11 16:17:47.017585 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.017484 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 11 16:17:47.071248 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.071219 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:47.071248 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.071243 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:47.071420 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:47.071321 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:47.071528 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:47.071488 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:47.184393 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.184360 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" event={"ID":"889874bb-c587-44d6-858f-207e25a501d6","Type":"ContainerStarted","Data":"4f18ccbad73872e6235f509981dbd150883835f255c899cf4db43a0f066fe377"} Sep 11 16:17:47.186369 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.186341 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gx7gz" event={"ID":"c3484cd6-fa9f-4e35-bfb7-15d16130f884","Type":"ContainerStarted","Data":"3e4a337c6772dc11f13398b03abee9296131c0148047ca315b486ef82e3c5060"} Sep 11 16:17:47.203201 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.203098 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lknf9" podStartSLOduration=4.546921412 podStartE2EDuration="24.203083227s" totalImagesPullingTime="19.656161815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:47.202433786 +0000 UTC m=+23.617531329" watchObservedRunningTime="2026-09-11 16:17:47.203083227 +0000 UTC m=+23.618180784" Sep 11 16:17:47.203595 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.203568 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tzp4p" podStartSLOduration=6.027031767 podStartE2EDuration="24.203559587s" totalImagesPullingTime="18.17652782s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.295766371 +0000 UTC m=+22.710863913" watchObservedRunningTime="2026-09-11 16:17:47.203559587 +0000 UTC m=+23.618657131" Sep 11 16:17:47.220627 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:47.220588 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gx7gz" podStartSLOduration=6.132965275 podStartE2EDuration="24.220574987s" totalImagesPullingTime="18.087609712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:47.220157411 +0000 UTC m=+23.635254956" watchObservedRunningTime="2026-09-11 16:17:47.220574987 +0000 UTC m=+23.635672529" Sep 11 16:17:48.071472 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:48.071388 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:48.071627 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:48.071546 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:48.193024 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:48.192986 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"0385ecf3d5523f93e57cda048cdf38cc7b3babcdf99e306c0bef485e50c1c3e3"} Sep 11 16:17:48.395396 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:48.395313 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:48.838550 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:48.838476 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:48.839166 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:48.839146 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:49.071393 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:49.071321 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:49.071607 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:49.071582 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:49.071751 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:49.071617 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:49.071751 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:49.071700 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:49.195945 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:49.195868 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6b2pn" Sep 11 16:17:50.071519 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:50.071487 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:50.071656 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:50.071632 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:50.199752 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:50.199666 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerStarted","Data":"2b3e307ff184f471be9362f1dd18309ef9e46fa230ef7b096bbbeb0fd822e40a"} Sep 11 16:17:50.205105 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:50.205075 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" event={"ID":"6cb0b991-2dce-4a62-b6a6-30e8dd32023d","Type":"ContainerStarted","Data":"fbf09186ccaf25accf368bca2b164ae45fa61bed4db7c6892a2b896a4697791b"} Sep 11 16:17:50.254397 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:50.254340 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" podStartSLOduration=9.145791623 podStartE2EDuration="27.254323921s" totalImagesPullingTime="18.108532298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:50.253089599 +0000 UTC m=+26.668187159" watchObservedRunningTime="2026-09-11 16:17:50.254323921 +0000 UTC m=+26.669421463" Sep 11 16:17:51.071774 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.071566 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:51.071916 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.071566 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:51.071916 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:51.071869 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:51.072005 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:51.071928 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:51.209224 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.209188 2553 generic.go:356] "Generic (PLEG): container finished" podID="a94516ff-ab02-4f35-82d4-6ac996fc5cb2" containerID="2b3e307ff184f471be9362f1dd18309ef9e46fa230ef7b096bbbeb0fd822e40a" exitCode=0 Sep 11 16:17:51.209630 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.209281 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerDied","Data":"2b3e307ff184f471be9362f1dd18309ef9e46fa230ef7b096bbbeb0fd822e40a"} Sep 11 16:17:51.210850 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.209939 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:51.210850 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.210014 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:51.210850 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.210061 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:51.225169 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.225148 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:51.225283 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.225254 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:17:51.990531 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.990427 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xrwjl"] Sep 11 16:17:51.990782 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.990617 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:51.990782 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:51.990746 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:51.991186 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.991158 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j7pkn"] Sep 11 16:17:51.991243 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.991231 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:51.991321 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:51.991304 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:51.991768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.991750 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5wxnc"] Sep 11 16:17:51.991843 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:51.991833 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:51.991925 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:51.991907 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:52.006271 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:52.006240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:52.107599 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:52.107561 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:52.213574 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:52.213542 2553 generic.go:356] "Generic (PLEG): container finished" podID="a94516ff-ab02-4f35-82d4-6ac996fc5cb2" containerID="08c25615cd80d7480f6ed0cc96d28fac0d76709107846af27e4f14ef470eb1c4" exitCode=0 Sep 11 16:17:52.214452 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:52.213587 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerDied","Data":"08c25615cd80d7480f6ed0cc96d28fac0d76709107846af27e4f14ef470eb1c4"} Sep 11 16:17:53.218079 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:53.217991 2553 generic.go:356] "Generic (PLEG): container finished" podID="a94516ff-ab02-4f35-82d4-6ac996fc5cb2" containerID="504a53e72bf80d60be6b55ff3e5d53d3ea6b65b50caf38980e279205df0bf95f" exitCode=0 Sep 11 16:17:53.218421 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:53.218088 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerDied","Data":"504a53e72bf80d60be6b55ff3e5d53d3ea6b65b50caf38980e279205df0bf95f"} Sep 11 16:17:54.072707 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.072659 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:54.072889 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:54.072790 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:54.072889 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.072797 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:54.072889 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:54.072876 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:54.073049 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.072897 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:54.073049 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:54.072961 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:54.125778 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.125674 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:54.188985 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.188796 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:54.488987 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.488895 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:54.553380 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:54.553348 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:56.072044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:56.072003 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:56.072044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:56.072033 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:56.072564 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:56.072002 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:56.072564 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:56.072140 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-j7pkn" podUID="172fbab1-39a1-4a58-bdab-94faf02f1526" Sep 11 16:17:56.072564 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:56.072267 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5wxnc" podUID="4ac947c7-0e15-4a76-88b8-0004ce8933e7" Sep 11 16:17:56.072564 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:56.072338 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xrwjl" podUID="b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09" Sep 11 16:17:57.248982 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.248952 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:57.319419 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.319382 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:17:57.703185 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.703157 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:57.703378 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.703279 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:57.703378 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.703343 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs podName:4ac947c7-0e15-4a76-88b8-0004ce8933e7 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:29.703322438 +0000 UTC m=+66.118419959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs") pod "network-metrics-daemon-5wxnc" (UID: "4ac947c7-0e15-4a76-88b8-0004ce8933e7") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 11 16:17:57.804235 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.804198 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:57.804393 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.804343 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:57.804431 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.804414 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret podName:b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:29.804396804 +0000 UTC m=+66.219494328 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret") pod "global-pull-secret-syncer-xrwjl" (UID: "b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09") : object "kube-system"/"original-pull-secret" not registered Sep 11 16:17:57.863146 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.863101 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-119.ec2.internal" event="NodeReady" Sep 11 16:17:57.863285 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.863254 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 11 16:17:57.904530 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.904500 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:57.904713 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.904659 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 11 16:17:57.904713 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.904703 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 11 16:17:57.904811 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.904717 2553 projected.go:196] Error preparing data for projected volume kube-api-access-s45cq for pod openshift-network-diagnostics/network-check-target-j7pkn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:57.904811 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:57.904782 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq podName:172fbab1-39a1-4a58-bdab-94faf02f1526 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:29.904762678 +0000 UTC m=+66.319860213 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-s45cq" (UniqueName: "kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq") pod "network-check-target-j7pkn" (UID: "172fbab1-39a1-4a58-bdab-94faf02f1526") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 11 16:17:57.918151 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.918128 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-72pf4"] Sep 11 16:17:57.940390 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.940364 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5dccd5bf64-gb52d"] Sep 11 16:17:57.940555 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.940487 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:57.946095 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.946068 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 11 16:17:57.947101 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.947052 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkwjt\"" Sep 11 16:17:57.947359 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.947341 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 11 16:17:57.956761 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.956716 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r7z8b"] Sep 11 16:17:57.956857 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.956841 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:57.966048 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.966022 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 11 16:17:57.966164 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.966090 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 11 16:17:57.967367 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.967152 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 11 16:17:57.967367 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.967191 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jml27\"" Sep 11 16:17:57.970884 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.970865 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 11 16:17:57.971887 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.971871 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-72pf4"] Sep 11 16:17:57.971946 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.971902 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dccd5bf64-gb52d"] Sep 11 16:17:57.971946 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.971922 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r7z8b"] Sep 11 16:17:57.972007 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.971976 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:57.974354 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.974337 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 11 16:17:57.974570 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.974555 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 11 16:17:57.974647 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.974639 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 11 16:17:57.975048 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:57.975033 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gkls5\"" Sep 11 16:17:58.071933 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.071908 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:17:58.072078 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.071932 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:17:58.072078 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.071969 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:17:58.076914 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.076867 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 16:17:58.076914 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.076907 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 16:17:58.076914 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.076872 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7kds9\"" Sep 11 16:17:58.077204 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.076934 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 16:17:58.077204 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.077024 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 16:17:58.077204 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.077149 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lgws6\"" Sep 11 16:17:58.105104 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105081 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwm55\" (UniqueName: \"kubernetes.io/projected/a167ebfd-9f52-46b9-b6c8-367d461a31d6-kube-api-access-cwm55\") pod \"ingress-canary-r7z8b\" (UID: \"a167ebfd-9f52-46b9-b6c8-367d461a31d6\") " pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.105104 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105105 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-image-registry-private-configuration\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105256 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105127 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebe454a6-746f-4012-a5a4-adbb251afcc4-ca-trust-extracted\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105256 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105149 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.105256 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105169 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09bf8587-6328-4fd6-8421-9b612669bc63-tmp-dir\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.105256 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105192 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-trusted-ca\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105256 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105247 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-certificates\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105474 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105282 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjxt8\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-kube-api-access-rjxt8\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105474 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105313 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/09bf8587-6328-4fd6-8421-9b612669bc63-config-volume\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.105474 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105338 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b58zw\" (UniqueName: \"kubernetes.io/projected/09bf8587-6328-4fd6-8421-9b612669bc63-kube-api-access-b58zw\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.105474 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105383 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105474 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105443 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-bound-sa-token\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105819 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105509 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-installation-pull-secrets\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.105819 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.105534 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a167ebfd-9f52-46b9-b6c8-367d461a31d6-cert\") pod \"ingress-canary-r7z8b\" (UID: \"a167ebfd-9f52-46b9-b6c8-367d461a31d6\") " pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.206726 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206670 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwm55\" (UniqueName: \"kubernetes.io/projected/a167ebfd-9f52-46b9-b6c8-367d461a31d6-kube-api-access-cwm55\") pod \"ingress-canary-r7z8b\" (UID: \"a167ebfd-9f52-46b9-b6c8-367d461a31d6\") " pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206726 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-image-registry-private-configuration\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206786 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebe454a6-746f-4012-a5a4-adbb251afcc4-ca-trust-extracted\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206810 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206834 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09bf8587-6328-4fd6-8421-9b612669bc63-tmp-dir\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206857 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-trusted-ca\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206882 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-certificates\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206903 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjxt8\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-kube-api-access-rjxt8\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206929 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/09bf8587-6328-4fd6-8421-9b612669bc63-config-volume\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206948 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b58zw\" (UniqueName: \"kubernetes.io/projected/09bf8587-6328-4fd6-8421-9b612669bc63-kube-api-access-b58zw\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.206979 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.206951 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.207040 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls podName:09bf8587-6328-4fd6-8421-9b612669bc63 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:58.707023059 +0000 UTC m=+35.122120580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls") pod "dns-default-72pf4" (UID: "09bf8587-6328-4fd6-8421-9b612669bc63") : secret "dns-default-metrics-tls" not found Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.206968 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207214 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-bound-sa-token\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207218 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09bf8587-6328-4fd6-8421-9b612669bc63-tmp-dir\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207254 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebe454a6-746f-4012-a5a4-adbb251afcc4-ca-trust-extracted\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207315 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-installation-pull-secrets\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.207331 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.207345 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dccd5bf64-gb52d: secret "image-registry-tls" not found Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207345 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a167ebfd-9f52-46b9-b6c8-367d461a31d6-cert\") pod \"ingress-canary-r7z8b\" (UID: \"a167ebfd-9f52-46b9-b6c8-367d461a31d6\") " pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.207463 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.207406 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls podName:ebe454a6-746f-4012-a5a4-adbb251afcc4 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:58.707388133 +0000 UTC m=+35.122485655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls") pod "image-registry-5dccd5bf64-gb52d" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4") : secret "image-registry-tls" not found Sep 11 16:17:58.208012 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207543 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/09bf8587-6328-4fd6-8421-9b612669bc63-config-volume\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.208012 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207607 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-certificates\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.208012 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.207885 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-trusted-ca\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.211207 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.211182 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a167ebfd-9f52-46b9-b6c8-367d461a31d6-cert\") pod \"ingress-canary-r7z8b\" (UID: \"a167ebfd-9f52-46b9-b6c8-367d461a31d6\") " pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.211305 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.211212 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-image-registry-private-configuration\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.211305 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.211242 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-installation-pull-secrets\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.219222 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.219200 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-bound-sa-token\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.219848 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.219820 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjxt8\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-kube-api-access-rjxt8\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.220178 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.220160 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwm55\" (UniqueName: \"kubernetes.io/projected/a167ebfd-9f52-46b9-b6c8-367d461a31d6-kube-api-access-cwm55\") pod \"ingress-canary-r7z8b\" (UID: \"a167ebfd-9f52-46b9-b6c8-367d461a31d6\") " pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.220386 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.220371 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b58zw\" (UniqueName: \"kubernetes.io/projected/09bf8587-6328-4fd6-8421-9b612669bc63-kube-api-access-b58zw\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.240182 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.240154 2553 generic.go:356] "Generic (PLEG): container finished" podID="a94516ff-ab02-4f35-82d4-6ac996fc5cb2" containerID="3dbf021c9350efd345ca79fb132892d8cfa963a97f90e522bdd4e91c93939033" exitCode=0 Sep 11 16:17:58.240291 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.240221 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerDied","Data":"3dbf021c9350efd345ca79fb132892d8cfa963a97f90e522bdd4e91c93939033"} Sep 11 16:17:58.283126 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.283102 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r7z8b" Sep 11 16:17:58.593885 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.593715 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r7z8b"] Sep 11 16:17:58.596475 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:17:58.596435 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda167ebfd_9f52_46b9_b6c8_367d461a31d6.slice/crio-54f5ec6f9fcfc6413588088aa1a87a57ba9534838df4d99cb7982b1146d9e151 WatchSource:0}: Error finding container 54f5ec6f9fcfc6413588088aa1a87a57ba9534838df4d99cb7982b1146d9e151: Status 404 returned error can't find the container with id 54f5ec6f9fcfc6413588088aa1a87a57ba9534838df4d99cb7982b1146d9e151 Sep 11 16:17:58.710644 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.710601 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:58.710881 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:58.710653 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:58.710881 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.710770 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 16:17:58.710881 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.710793 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 16:17:58.710881 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.710808 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dccd5bf64-gb52d: secret "image-registry-tls" not found Sep 11 16:17:58.710881 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.710834 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls podName:09bf8587-6328-4fd6-8421-9b612669bc63 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.710818005 +0000 UTC m=+36.125915526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls") pod "dns-default-72pf4" (UID: "09bf8587-6328-4fd6-8421-9b612669bc63") : secret "dns-default-metrics-tls" not found Sep 11 16:17:58.710881 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:58.710855 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls podName:ebe454a6-746f-4012-a5a4-adbb251afcc4 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.710840681 +0000 UTC m=+36.125938203 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls") pod "image-registry-5dccd5bf64-gb52d" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4") : secret "image-registry-tls" not found Sep 11 16:17:59.246450 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:59.246413 2553 generic.go:356] "Generic (PLEG): container finished" podID="a94516ff-ab02-4f35-82d4-6ac996fc5cb2" containerID="c09f7a5ebd617a7b65b6fe8720b0397a4c9fe809af24e906640c7f4381f24c22" exitCode=0 Sep 11 16:17:59.246715 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:59.246502 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerDied","Data":"c09f7a5ebd617a7b65b6fe8720b0397a4c9fe809af24e906640c7f4381f24c22"} Sep 11 16:17:59.247881 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:59.247869 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r7z8b" event={"ID":"a167ebfd-9f52-46b9-b6c8-367d461a31d6","Type":"ContainerStarted","Data":"54f5ec6f9fcfc6413588088aa1a87a57ba9534838df4d99cb7982b1146d9e151"} Sep 11 16:17:59.718635 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:59.718594 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:17:59.719381 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:17:59.718652 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:17:59.719381 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:59.718800 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 16:17:59.719381 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:59.718833 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 16:17:59.719381 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:59.718846 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dccd5bf64-gb52d: secret "image-registry-tls" not found Sep 11 16:17:59.719381 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:59.718881 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls podName:09bf8587-6328-4fd6-8421-9b612669bc63 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:01.718857794 +0000 UTC m=+38.133955334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls") pod "dns-default-72pf4" (UID: "09bf8587-6328-4fd6-8421-9b612669bc63") : secret "dns-default-metrics-tls" not found Sep 11 16:17:59.719381 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:17:59.718901 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls podName:ebe454a6-746f-4012-a5a4-adbb251afcc4 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:01.7188914 +0000 UTC m=+38.133988926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls") pod "image-registry-5dccd5bf64-gb52d" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4") : secret "image-registry-tls" not found Sep 11 16:18:00.254447 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:00.254412 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g24b" event={"ID":"a94516ff-ab02-4f35-82d4-6ac996fc5cb2","Type":"ContainerStarted","Data":"6cc3d63e956110bb4cbdae998ffbf4a55a1b3f8b0aa6bfeb3f6a08d102622f90"} Sep 11 16:18:00.281000 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:00.280200 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6g24b" podStartSLOduration=9.388938656 podStartE2EDuration="37.280179291s" totalImagesPullingTime="27.891240635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:00.279875759 +0000 UTC m=+36.694973326" watchObservedRunningTime="2026-09-11 16:18:00.280179291 +0000 UTC m=+36.695276831" Sep 11 16:18:00.438432 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:00.438393 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:00.531543 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:00.531462 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:00.607480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:00.607448 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:01.186234 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.186192 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:01.259003 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.258963 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r7z8b" event={"ID":"a167ebfd-9f52-46b9-b6c8-367d461a31d6","Type":"ContainerStarted","Data":"afbb82a69de631fcd9c2bbd775576b17a67185bfd5d62677092f4746384061fd"} Sep 11 16:18:01.276379 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.276328 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r7z8b" podStartSLOduration=1.8092694360000001 podStartE2EDuration="4.276311667s" totalImagesPullingTime="2.467042231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:01.275660758 +0000 UTC m=+37.690758302" watchObservedRunningTime="2026-09-11 16:18:01.276311667 +0000 UTC m=+37.691409210" Sep 11 16:18:01.280832 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.280803 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:01.430019 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.429944 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x7fwg"] Sep 11 16:18:01.434198 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.434167 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:01.436461 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.436445 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.439477 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.439458 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 11 16:18:01.439596 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.439579 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 11 16:18:01.439663 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.439583 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 11 16:18:01.439742 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.439698 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ddmk9\"" Sep 11 16:18:01.439849 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.439834 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 11 16:18:01.455136 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.455114 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x7fwg"] Sep 11 16:18:01.522582 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.522559 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:01.532641 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.532621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6e4169a0-5b02-4c7b-912c-b3f944d03f43-crio-socket\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.532750 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.532665 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e4169a0-5b02-4c7b-912c-b3f944d03f43-data-volume\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.532750 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.532725 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.532818 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.532753 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh65v\" (UniqueName: \"kubernetes.io/projected/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-api-access-dh65v\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.532818 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.532803 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634143 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634106 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e4169a0-5b02-4c7b-912c-b3f944d03f43-data-volume\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634143 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634141 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634326 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634162 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh65v\" (UniqueName: \"kubernetes.io/projected/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-api-access-dh65v\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634326 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634214 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634326 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.634268 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 16:18:01.634326 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634303 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6e4169a0-5b02-4c7b-912c-b3f944d03f43-crio-socket\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634466 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.634351 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.134330424 +0000 UTC m=+38.549427962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : configmap "kube-rbac-proxy" not found Sep 11 16:18:01.634466 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.634432 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 16:18:01.634532 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.634490 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.134475993 +0000 UTC m=+38.549573515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : secret "insights-runtime-extractor-tls" not found Sep 11 16:18:01.634532 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634497 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e4169a0-5b02-4c7b-912c-b3f944d03f43-data-volume\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.634601 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.634557 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6e4169a0-5b02-4c7b-912c-b3f944d03f43-crio-socket\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.646430 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.646401 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh65v\" (UniqueName: \"kubernetes.io/projected/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-api-access-dh65v\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:01.734973 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.734928 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:01.734973 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.734972 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:01.735152 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.735091 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 16:18:01.735152 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.735095 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 16:18:01.735152 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.735112 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dccd5bf64-gb52d: secret "image-registry-tls" not found Sep 11 16:18:01.735243 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.735161 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls podName:09bf8587-6328-4fd6-8421-9b612669bc63 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.735142704 +0000 UTC m=+42.150240241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls") pod "dns-default-72pf4" (UID: "09bf8587-6328-4fd6-8421-9b612669bc63") : secret "dns-default-metrics-tls" not found Sep 11 16:18:01.735243 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:01.735176 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls podName:ebe454a6-746f-4012-a5a4-adbb251afcc4 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.735169096 +0000 UTC m=+42.150266617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls") pod "image-registry-5dccd5bf64-gb52d" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4") : secret "image-registry-tls" not found Sep 11 16:18:01.820976 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.820936 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:01.921969 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:01.921942 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:02.137920 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.137663 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:02.137920 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.137910 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:02.138141 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:02.137785 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 16:18:02.138141 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:02.137985 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:03.13797018 +0000 UTC m=+39.553067700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : configmap "kube-rbac-proxy" not found Sep 11 16:18:02.138141 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:02.138052 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 16:18:02.138141 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:02.138108 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:03.138093158 +0000 UTC m=+39.553190697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : secret "insights-runtime-extractor-tls" not found Sep 11 16:18:02.256489 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.256456 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:02.348144 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.348100 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:02.421851 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.421767 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:02.501108 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.501078 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:02.590550 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:02.590518 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:03.146422 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:03.146386 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:03.146592 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:03.146441 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:03.146592 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:03.146520 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 16:18:03.146592 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:03.146567 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 16:18:03.146592 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:03.146585 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.146568751 +0000 UTC m=+41.561666271 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : configmap "kube-rbac-proxy" not found Sep 11 16:18:03.146759 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:03.146605 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.146594374 +0000 UTC m=+41.561691918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : secret "insights-runtime-extractor-tls" not found Sep 11 16:18:03.153561 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:03.153543 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bqpcw_0475e5f9-fcb0-4165-b925-ca7901a3b53d/dns-node-resolver/0.log" Sep 11 16:18:03.222450 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:03.222419 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:04.029478 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:04.029446 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:04.126824 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:04.126790 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:04.153155 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:04.153124 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-62tgl_1a762e41-b42c-470f-bd77-4d790aed8532/node-ca/0.log" Sep 11 16:18:04.194889 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:04.194851 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:05.156752 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:05.156722 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r7z8b_a167ebfd-9f52-46b9-b6c8-367d461a31d6/serve-healthcheck-canary/0.log" Sep 11 16:18:05.160856 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:05.160836 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:05.160940 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:05.160925 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:05.160994 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.160980 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 16:18:05.161027 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.161002 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 11 16:18:05.161059 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.161047 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:09.161034432 +0000 UTC m=+45.576131953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : configmap "kube-rbac-proxy" not found Sep 11 16:18:05.161099 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.161059 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:09.161053466 +0000 UTC m=+45.576150987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : secret "insights-runtime-extractor-tls" not found Sep 11 16:18:05.189165 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:05.189141 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:05.764591 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:05.764556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:05.764591 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:05.764592 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:05.764821 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.764738 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 11 16:18:05.764821 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.764798 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls podName:09bf8587-6328-4fd6-8421-9b612669bc63 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:13.764783652 +0000 UTC m=+50.179881172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls") pod "dns-default-72pf4" (UID: "09bf8587-6328-4fd6-8421-9b612669bc63") : secret "dns-default-metrics-tls" not found Sep 11 16:18:05.764906 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.764740 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 11 16:18:05.764906 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.764839 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5dccd5bf64-gb52d: secret "image-registry-tls" not found Sep 11 16:18:05.764906 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:05.764881 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls podName:ebe454a6-746f-4012-a5a4-adbb251afcc4 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:13.76486972 +0000 UTC m=+50.179967241 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls") pod "image-registry-5dccd5bf64-gb52d" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4") : secret "image-registry-tls" not found Sep 11 16:18:07.931859 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:07.931828 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:08.348627 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:08.348593 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:08.453765 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:08.453734 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:08.669143 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:08.669066 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:08.768899 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:08.768869 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:09.149118 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.149090 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:09.190188 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.190157 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:09.190324 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.190208 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:09.190376 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:09.190362 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 11 16:18:09.190440 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:09.190431 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls podName:6e4169a0-5b02-4c7b-912c-b3f944d03f43 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:17.190413452 +0000 UTC m=+53.605510994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7fwg" (UID: "6e4169a0-5b02-4c7b-912c-b3f944d03f43") : secret "insights-runtime-extractor-tls" not found Sep 11 16:18:09.190736 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.190718 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6e4169a0-5b02-4c7b-912c-b3f944d03f43-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:09.247926 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.247892 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:09.327700 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.327657 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:09.819067 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:09.819037 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:10.405735 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:10.405697 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:10.613043 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:10.613014 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:10.807346 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:10.807317 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:11.004242 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:11.004213 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:12.004001 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:12.003967 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:12.063303 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:12.063270 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:12.359084 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:12.358995 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:12.444791 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:12.444757 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:12.607356 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:12.607327 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:12.798201 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:12.798170 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:13.228995 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.228964 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:13.591935 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.591857 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:13.692495 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.692457 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:13.792227 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.792195 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:13.825872 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.825850 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:13.825957 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.825884 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:13.828731 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.828700 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09bf8587-6328-4fd6-8421-9b612669bc63-metrics-tls\") pod \"dns-default-72pf4\" (UID: \"09bf8587-6328-4fd6-8421-9b612669bc63\") " pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:13.828839 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.828780 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"image-registry-5dccd5bf64-gb52d\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:13.851095 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.851042 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:13.869449 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.869429 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:13.892306 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.892269 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:13.994058 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:13.994022 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.147497 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.147423 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-72pf4"] Sep 11 16:18:14.150724 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:14.150698 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bf8587_6328_4fd6_8421_9b612669bc63.slice/crio-76ac00103f385b404424ae4df18d4b5b59128b709dcbc7e30992207b97d6cb79 WatchSource:0}: Error finding container 76ac00103f385b404424ae4df18d4b5b59128b709dcbc7e30992207b97d6cb79: Status 404 returned error can't find the container with id 76ac00103f385b404424ae4df18d4b5b59128b709dcbc7e30992207b97d6cb79 Sep 11 16:18:14.175044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.175018 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5dccd5bf64-gb52d"] Sep 11 16:18:14.177722 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:14.177698 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe454a6_746f_4012_a5a4_adbb251afcc4.slice/crio-df116542c98931fe06b4066eed7f83ed1e86055b1012a37f1decb668ff83569c WatchSource:0}: Error finding container df116542c98931fe06b4066eed7f83ed1e86055b1012a37f1decb668ff83569c: Status 404 returned error can't find the container with id df116542c98931fe06b4066eed7f83ed1e86055b1012a37f1decb668ff83569c Sep 11 16:18:14.197002 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.196975 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.292817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.292790 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" event={"ID":"ebe454a6-746f-4012-a5a4-adbb251afcc4","Type":"ContainerStarted","Data":"df116542c98931fe06b4066eed7f83ed1e86055b1012a37f1decb668ff83569c"} Sep 11 16:18:14.293912 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.293892 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-72pf4" event={"ID":"09bf8587-6328-4fd6-8421-9b612669bc63","Type":"ContainerStarted","Data":"76ac00103f385b404424ae4df18d4b5b59128b709dcbc7e30992207b97d6cb79"} Sep 11 16:18:14.298666 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.298642 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.399443 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.399354 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.500318 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.500275 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.601463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.601411 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.710226 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.710188 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.805278 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.805237 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:14.903200 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:14.903162 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:15.207058 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.207030 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:15.298809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.298772 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" event={"ID":"ebe454a6-746f-4012-a5a4-adbb251afcc4","Type":"ContainerStarted","Data":"1bd8d35bbe478909d426d635386221e034b9cc74baa33d283400ec096a3260af"} Sep 11 16:18:15.299314 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.299286 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:15.307468 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.307437 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:15.352765 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.352714 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" podStartSLOduration=26.352673207 podStartE2EDuration="26.352673207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:15.3519096 +0000 UTC m=+51.767007146" watchObservedRunningTime="2026-09-11 16:18:15.352673207 +0000 UTC m=+51.767770753" Sep 11 16:18:15.409380 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.409341 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:15.512299 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.512217 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:15.934255 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:15.934172 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:16.040704 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.040654 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:16.228190 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.228160 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:16.303378 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.303343 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-72pf4" event={"ID":"09bf8587-6328-4fd6-8421-9b612669bc63","Type":"ContainerStarted","Data":"354a88ad633edc2ca10ed406efe25ecab1db0b3fa8dcd4ba3c1a0f5e6ec184cc"} Sep 11 16:18:16.303847 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.303393 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-72pf4" event={"ID":"09bf8587-6328-4fd6-8421-9b612669bc63","Type":"ContainerStarted","Data":"973a06fefa2e0ed0258f65c27536c544b7139614c2f6ff1c23be51ce4f8aa3d9"} Sep 11 16:18:16.303847 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.303540 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:16.322007 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.321959 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-72pf4" podStartSLOduration=17.935314728 podStartE2EDuration="19.321942683s" totalImagesPullingTime="1.386627955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:16.321660744 +0000 UTC m=+52.736758288" watchObservedRunningTime="2026-09-11 16:18:16.321942683 +0000 UTC m=+52.737040223" Sep 11 16:18:16.616228 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.616139 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:16.800985 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:16.800950 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:17.253612 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:17.253573 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:17.256044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:17.256021 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6e4169a0-5b02-4c7b-912c-b3f944d03f43-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7fwg\" (UID: \"6e4169a0-5b02-4c7b-912c-b3f944d03f43\") " pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:17.345694 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:17.345634 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x7fwg" Sep 11 16:18:17.411534 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:17.411440 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:17.511136 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:17.511056 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:17.618331 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:17.618295 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x7fwg"] Sep 11 16:18:17.623106 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:17.623082 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4169a0_5b02_4c7b_912c_b3f944d03f43.slice/crio-8b757a257b0d6a967a5cc643f7c92ef9fc7ce6fa8df200fc485b93f5e0ea7ed8 WatchSource:0}: Error finding container 8b757a257b0d6a967a5cc643f7c92ef9fc7ce6fa8df200fc485b93f5e0ea7ed8: Status 404 returned error can't find the container with id 8b757a257b0d6a967a5cc643f7c92ef9fc7ce6fa8df200fc485b93f5e0ea7ed8 Sep 11 16:18:18.016178 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.016148 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:18.207971 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.207937 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:18.310791 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.310765 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7fwg" event={"ID":"6e4169a0-5b02-4c7b-912c-b3f944d03f43","Type":"ContainerStarted","Data":"04209e7e5068775f513f1306322a4809dedcbb55754eb40eac01c9817ed0f32c"} Sep 11 16:18:18.310893 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.310842 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7fwg" event={"ID":"6e4169a0-5b02-4c7b-912c-b3f944d03f43","Type":"ContainerStarted","Data":"2df39a9157b882578dde4bfe47c248f51f4763314a05d2c80096d5a5e4cfa870"} Sep 11 16:18:18.310893 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.310863 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7fwg" event={"ID":"6e4169a0-5b02-4c7b-912c-b3f944d03f43","Type":"ContainerStarted","Data":"8b757a257b0d6a967a5cc643f7c92ef9fc7ce6fa8df200fc485b93f5e0ea7ed8"} Sep 11 16:18:18.407878 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.407842 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:18.803439 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:18.803403 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:19.202293 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:19.202263 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:19.410518 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:19.410487 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:20.319755 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:20.319720 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7fwg" event={"ID":"6e4169a0-5b02-4c7b-912c-b3f944d03f43","Type":"ContainerStarted","Data":"eee92072451dcd2b5aaebc01c881bf32d79eaf442792cf3d4c5e3f373e56d84d"} Sep 11 16:18:20.345020 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:20.344974 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x7fwg" podStartSLOduration=17.38903845 podStartE2EDuration="19.344958623s" totalImagesPullingTime="1.955920173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:18:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:20.343947699 +0000 UTC m=+56.759045242" watchObservedRunningTime="2026-09-11 16:18:20.344958623 +0000 UTC m=+56.760056166" Sep 11 16:18:23.229469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:23.229438 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bdf94" Sep 11 16:18:24.126283 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:24.126247 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:24.197588 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:24.197558 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:26.217378 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.217342 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4"] Sep 11 16:18:26.220055 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.220034 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs"] Sep 11 16:18:26.220187 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.220143 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.221740 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.221721 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.223908 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.223893 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 11 16:18:26.224139 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.224124 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 11 16:18:26.225455 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.225433 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 11 16:18:26.225455 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.225435 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 11 16:18:26.225592 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.225529 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 11 16:18:26.225926 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.225904 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fvnsk\"" Sep 11 16:18:26.228059 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.228040 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dccd5bf64-gb52d"] Sep 11 16:18:26.243436 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.243410 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4"] Sep 11 16:18:26.244166 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.244143 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs"] Sep 11 16:18:26.307961 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.307935 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-72pf4" Sep 11 16:18:26.320545 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.320519 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdqv8\" (UniqueName: \"kubernetes.io/projected/5fd77f1b-e80f-462b-a32f-57f8e3624d18-kube-api-access-pdqv8\") pod \"managed-serviceaccount-addon-agent-58c6c95d5b-djcq4\" (UID: \"5fd77f1b-e80f-462b-a32f-57f8e3624d18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.320757 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.320737 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf85l\" (UniqueName: \"kubernetes.io/projected/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-kube-api-access-gf85l\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.320803 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.320773 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-tmp\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.320838 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.320805 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-klusterlet-config\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.320883 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.320837 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fd77f1b-e80f-462b-a32f-57f8e3624d18-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58c6c95d5b-djcq4\" (UID: \"5fd77f1b-e80f-462b-a32f-57f8e3624d18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.323001 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.322971 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-7mr9t"] Sep 11 16:18:26.324735 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.324721 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:26.329497 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.329479 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 11 16:18:26.329851 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.329830 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5h7m2\"" Sep 11 16:18:26.329976 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.329954 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 11 16:18:26.353547 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.353517 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-7mr9t"] Sep 11 16:18:26.367982 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.367913 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:26.421786 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.421757 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js77b\" (UniqueName: \"kubernetes.io/projected/9e992fc4-f472-48bc-b9fa-bccf95275e45-kube-api-access-js77b\") pod \"downloads-76f99584f4-7mr9t\" (UID: \"9e992fc4-f472-48bc-b9fa-bccf95275e45\") " pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:26.421944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.421798 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf85l\" (UniqueName: \"kubernetes.io/projected/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-kube-api-access-gf85l\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.421944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.421819 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-tmp\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.421944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.421882 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-klusterlet-config\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.421944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.421905 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fd77f1b-e80f-462b-a32f-57f8e3624d18-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58c6c95d5b-djcq4\" (UID: \"5fd77f1b-e80f-462b-a32f-57f8e3624d18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.421944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.421945 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdqv8\" (UniqueName: \"kubernetes.io/projected/5fd77f1b-e80f-462b-a32f-57f8e3624d18-kube-api-access-pdqv8\") pod \"managed-serviceaccount-addon-agent-58c6c95d5b-djcq4\" (UID: \"5fd77f1b-e80f-462b-a32f-57f8e3624d18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.422235 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.422215 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-tmp\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.424454 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.424432 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-klusterlet-config\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.424644 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.424625 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fd77f1b-e80f-462b-a32f-57f8e3624d18-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-58c6c95d5b-djcq4\" (UID: \"5fd77f1b-e80f-462b-a32f-57f8e3624d18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.433116 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.433091 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf85l\" (UniqueName: \"kubernetes.io/projected/fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6-kube-api-access-gf85l\") pod \"klusterlet-addon-workmgr-dfb6fb9c8-hv7zs\" (UID: \"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.434558 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.434537 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdqv8\" (UniqueName: \"kubernetes.io/projected/5fd77f1b-e80f-462b-a32f-57f8e3624d18-kube-api-access-pdqv8\") pod \"managed-serviceaccount-addon-agent-58c6c95d5b-djcq4\" (UID: \"5fd77f1b-e80f-462b-a32f-57f8e3624d18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.468464 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.468394 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:26.522517 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.522475 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js77b\" (UniqueName: \"kubernetes.io/projected/9e992fc4-f472-48bc-b9fa-bccf95275e45-kube-api-access-js77b\") pod \"downloads-76f99584f4-7mr9t\" (UID: \"9e992fc4-f472-48bc-b9fa-bccf95275e45\") " pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:26.532502 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.532481 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js77b\" (UniqueName: \"kubernetes.io/projected/9e992fc4-f472-48bc-b9fa-bccf95275e45-kube-api-access-js77b\") pod \"downloads-76f99584f4-7mr9t\" (UID: \"9e992fc4-f472-48bc-b9fa-bccf95275e45\") " pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:26.539275 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.539251 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" Sep 11 16:18:26.545804 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.545785 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:26.591160 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.591125 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:26.633324 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.633296 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:26.777702 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.777591 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:26.818971 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.818945 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4"] Sep 11 16:18:26.821668 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:26.821646 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fd77f1b_e80f_462b_a32f_57f8e3624d18.slice/crio-7d88286ab6018ee480f1823fff4116addf9b284aa0c8f5748a5739bbd316f2a3 WatchSource:0}: Error finding container 7d88286ab6018ee480f1823fff4116addf9b284aa0c8f5748a5739bbd316f2a3: Status 404 returned error can't find the container with id 7d88286ab6018ee480f1823fff4116addf9b284aa0c8f5748a5739bbd316f2a3 Sep 11 16:18:26.840435 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.840408 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs"] Sep 11 16:18:26.843693 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:26.843658 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc8d9f4a_1eea_4e2d_aad6_e880b0b073d6.slice/crio-2e0e92a7766bdaa77802a8aa3ce70695ced3af017842700693de3cf5497786c0 WatchSource:0}: Error finding container 2e0e92a7766bdaa77802a8aa3ce70695ced3af017842700693de3cf5497786c0: Status 404 returned error can't find the container with id 2e0e92a7766bdaa77802a8aa3ce70695ced3af017842700693de3cf5497786c0 Sep 11 16:18:26.911233 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:26.911206 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-7mr9t"] Sep 11 16:18:26.914434 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:26.914410 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e992fc4_f472_48bc_b9fa_bccf95275e45.slice/crio-a44718e53a7ff28162123ca636c1b013361d53c4466e5a0e2456e5d065579150 WatchSource:0}: Error finding container a44718e53a7ff28162123ca636c1b013361d53c4466e5a0e2456e5d065579150: Status 404 returned error can't find the container with id a44718e53a7ff28162123ca636c1b013361d53c4466e5a0e2456e5d065579150 Sep 11 16:18:27.345696 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:27.345591 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" event={"ID":"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6","Type":"ContainerStarted","Data":"2e0e92a7766bdaa77802a8aa3ce70695ced3af017842700693de3cf5497786c0"} Sep 11 16:18:27.348176 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:27.348145 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-7mr9t" event={"ID":"9e992fc4-f472-48bc-b9fa-bccf95275e45","Type":"ContainerStarted","Data":"a44718e53a7ff28162123ca636c1b013361d53c4466e5a0e2456e5d065579150"} Sep 11 16:18:27.349657 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:27.349630 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" event={"ID":"5fd77f1b-e80f-462b-a32f-57f8e3624d18","Type":"ContainerStarted","Data":"7d88286ab6018ee480f1823fff4116addf9b284aa0c8f5748a5739bbd316f2a3"} Sep 11 16:18:29.363968 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.363901 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cfc5f8858-r58xm"] Sep 11 16:18:29.367060 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.366162 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.370374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.370321 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 11 16:18:29.370374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.370335 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5nx5t\"" Sep 11 16:18:29.370374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.370325 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 11 16:18:29.370665 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.370644 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 11 16:18:29.371089 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.370978 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 11 16:18:29.371762 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.371733 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 11 16:18:29.378550 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.377747 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cfc5f8858-r58xm"] Sep 11 16:18:29.450909 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.450875 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-service-ca\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.451077 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.450951 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-oauth-serving-cert\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.451077 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.451024 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-oauth-config\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.451077 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.451051 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-serving-cert\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.451240 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.451080 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29swq\" (UniqueName: \"kubernetes.io/projected/665ee9f0-e0ed-4824-a479-bd366d312452-kube-api-access-29swq\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.451240 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.451117 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-console-config\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.552274 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.552235 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-oauth-serving-cert\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.552464 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.552318 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-oauth-config\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.552464 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.552336 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-serving-cert\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.552464 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.552357 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29swq\" (UniqueName: \"kubernetes.io/projected/665ee9f0-e0ed-4824-a479-bd366d312452-kube-api-access-29swq\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.552464 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.552389 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-console-config\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.552777 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.552749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-service-ca\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.553130 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.553078 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-oauth-serving-cert\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.553237 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.553169 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-console-config\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.553391 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.553359 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-service-ca\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.555781 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.555736 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-oauth-config\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.556026 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.556001 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-serving-cert\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.564397 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.564374 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29swq\" (UniqueName: \"kubernetes.io/projected/665ee9f0-e0ed-4824-a479-bd366d312452-kube-api-access-29swq\") pod \"console-6cfc5f8858-r58xm\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.679847 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.679769 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:29.754802 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.754761 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:18:29.758378 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.758342 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 11 16:18:29.776131 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.776102 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ac947c7-0e15-4a76-88b8-0004ce8933e7-metrics-certs\") pod \"network-metrics-daemon-5wxnc\" (UID: \"4ac947c7-0e15-4a76-88b8-0004ce8933e7\") " pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:18:29.855735 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.855366 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:18:29.858792 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.858765 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 11 16:18:29.868507 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.868484 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09-original-pull-secret\") pod \"global-pull-secret-syncer-xrwjl\" (UID: \"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09\") " pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:18:29.911333 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.911299 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7kds9\"" Sep 11 16:18:29.918765 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.918740 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5wxnc" Sep 11 16:18:29.921041 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.921016 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xrwjl" Sep 11 16:18:29.956381 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.956346 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:18:29.959613 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.959409 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 11 16:18:29.970798 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.970566 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 11 16:18:29.980919 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:29.980891 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s45cq\" (UniqueName: \"kubernetes.io/projected/172fbab1-39a1-4a58-bdab-94faf02f1526-kube-api-access-s45cq\") pod \"network-check-target-j7pkn\" (UID: \"172fbab1-39a1-4a58-bdab-94faf02f1526\") " pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:18:30.130131 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.130065 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:30.218091 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.218003 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lgws6\"" Sep 11 16:18:30.225602 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.225574 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:18:30.231648 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.231614 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:30.325889 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.325856 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:30.675789 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.675709 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:30.776324 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.776238 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:30.877636 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:30.877547 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:31.013059 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:31.013022 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:31.115471 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:31.115437 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:32.306243 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.306150 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xrwjl"] Sep 11 16:18:32.310177 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:32.310145 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0c5ba56_2f15_44c5_bf94_d48bb6dd4f09.slice/crio-fcfd13b652d7be4f373464dd8d3577019ec77e4d4acd6323fe3bb54b0322c04a WatchSource:0}: Error finding container fcfd13b652d7be4f373464dd8d3577019ec77e4d4acd6323fe3bb54b0322c04a: Status 404 returned error can't find the container with id fcfd13b652d7be4f373464dd8d3577019ec77e4d4acd6323fe3bb54b0322c04a Sep 11 16:18:32.311702 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.311629 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cfc5f8858-r58xm"] Sep 11 16:18:32.313959 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:32.313940 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod665ee9f0_e0ed_4824_a479_bd366d312452.slice/crio-4306ec61c9cfdf93cb150e482c7ed94ec6bdeb564a62f916c938586c44992d0b WatchSource:0}: Error finding container 4306ec61c9cfdf93cb150e482c7ed94ec6bdeb564a62f916c938586c44992d0b: Status 404 returned error can't find the container with id 4306ec61c9cfdf93cb150e482c7ed94ec6bdeb564a62f916c938586c44992d0b Sep 11 16:18:32.347854 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.347825 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-j7pkn"] Sep 11 16:18:32.352865 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:32.352832 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod172fbab1_39a1_4a58_bdab_94faf02f1526.slice/crio-f3a071b8665a9c0451528c5e00a998c092b863ab733677d3cb812a9eb16e07d3 WatchSource:0}: Error finding container f3a071b8665a9c0451528c5e00a998c092b863ab733677d3cb812a9eb16e07d3: Status 404 returned error can't find the container with id f3a071b8665a9c0451528c5e00a998c092b863ab733677d3cb812a9eb16e07d3 Sep 11 16:18:32.373768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.373309 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" event={"ID":"fc8d9f4a-1eea-4e2d-aad6-e880b0b073d6","Type":"ContainerStarted","Data":"0b3931844ec319255587ae0c299898907248a478208c3a047be41edcf8e76c6e"} Sep 11 16:18:32.373768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.373370 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5wxnc"] Sep 11 16:18:32.373768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.373405 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:32.375415 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.375393 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" Sep 11 16:18:32.375957 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.375934 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" event={"ID":"5fd77f1b-e80f-462b-a32f-57f8e3624d18","Type":"ContainerStarted","Data":"62972732370ef59deff66803ab0ed96eeb82442cf0617ffe335da1e7eb8606ad"} Sep 11 16:18:32.376280 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:32.376257 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac947c7_0e15_4a76_88b8_0004ce8933e7.slice/crio-a4b44f18da705870bc6beb63080ca71b5af5875ba9efea98bdb5ba835fb6348b WatchSource:0}: Error finding container a4b44f18da705870bc6beb63080ca71b5af5875ba9efea98bdb5ba835fb6348b: Status 404 returned error can't find the container with id a4b44f18da705870bc6beb63080ca71b5af5875ba9efea98bdb5ba835fb6348b Sep 11 16:18:32.377498 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.377463 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j7pkn" event={"ID":"172fbab1-39a1-4a58-bdab-94faf02f1526","Type":"ContainerStarted","Data":"f3a071b8665a9c0451528c5e00a998c092b863ab733677d3cb812a9eb16e07d3"} Sep 11 16:18:32.378954 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.378929 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xrwjl" event={"ID":"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09","Type":"ContainerStarted","Data":"fcfd13b652d7be4f373464dd8d3577019ec77e4d4acd6323fe3bb54b0322c04a"} Sep 11 16:18:32.380604 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.380583 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfc5f8858-r58xm" event={"ID":"665ee9f0-e0ed-4824-a479-bd366d312452","Type":"ContainerStarted","Data":"4306ec61c9cfdf93cb150e482c7ed94ec6bdeb564a62f916c938586c44992d0b"} Sep 11 16:18:32.433083 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.433019 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-dfb6fb9c8-hv7zs" podStartSLOduration=1.233546858 podStartE2EDuration="6.433000543s" totalImagesPullingTime="5.199453685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:32.404128868 +0000 UTC m=+68.819226405" watchObservedRunningTime="2026-09-11 16:18:32.433000543 +0000 UTC m=+68.848098086" Sep 11 16:18:32.451743 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.451664 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-58c6c95d5b-djcq4" podStartSLOduration=1.257023301 podStartE2EDuration="6.451642744s" totalImagesPullingTime="5.194619443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:32.450743226 +0000 UTC m=+68.865840773" watchObservedRunningTime="2026-09-11 16:18:32.451642744 +0000 UTC m=+68.866740290" Sep 11 16:18:32.929390 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:32.929288 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:33.011013 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:33.010919 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:33.164750 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:33.164604 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:33.267259 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:33.267103 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:33.369150 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:33.369070 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:33.388532 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:33.388464 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5wxnc" event={"ID":"4ac947c7-0e15-4a76-88b8-0004ce8933e7","Type":"ContainerStarted","Data":"a4b44f18da705870bc6beb63080ca71b5af5875ba9efea98bdb5ba835fb6348b"} Sep 11 16:18:33.432510 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:33.432479 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:34.351380 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.351300 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:34.389946 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.389920 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-7glw9"] Sep 11 16:18:34.393836 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.393805 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.399421 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.397787 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 11 16:18:34.399421 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.399173 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 11 16:18:34.399617 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.399424 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 11 16:18:34.399675 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.399615 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 11 16:18:34.400785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.399843 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 11 16:18:34.400785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.400052 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nqllk\"" Sep 11 16:18:34.400785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.400321 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5wxnc" event={"ID":"4ac947c7-0e15-4a76-88b8-0004ce8933e7","Type":"ContainerStarted","Data":"05a5bbd3bc00f25156e4cb0c8e3ce738a8452bd427f67922c4db3aae2e17804a"} Sep 11 16:18:34.400785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.400361 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5wxnc" event={"ID":"4ac947c7-0e15-4a76-88b8-0004ce8933e7","Type":"ContainerStarted","Data":"388c095be290cca44ed661d290e58fc194f4272f6ac5734c58fb9256bfe5e768"} Sep 11 16:18:34.405307 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.405286 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-7glw9"] Sep 11 16:18:34.426979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.426883 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:34.437509 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.436788 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5wxnc" podStartSLOduration=70.063503989 podStartE2EDuration="1m11.436766775s" totalImagesPullingTime="1.373262786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:34.434969786 +0000 UTC m=+70.850067329" watchObservedRunningTime="2026-09-11 16:18:34.436766775 +0000 UTC m=+70.851864316" Sep 11 16:18:34.498081 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.498046 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.498321 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.498124 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx7wl\" (UniqueName: \"kubernetes.io/projected/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-kube-api-access-jx7wl\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.498467 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.498423 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.498589 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.498479 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.515428 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.515282 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:34.586845 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.586672 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:34.600132 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.599526 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx7wl\" (UniqueName: \"kubernetes.io/projected/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-kube-api-access-jx7wl\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.600132 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.599598 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.600132 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.599638 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.600132 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.600054 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.600583 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.600534 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.621219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.621149 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.622205 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.622180 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.629918 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.629830 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx7wl\" (UniqueName: \"kubernetes.io/projected/2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa-kube-api-access-jx7wl\") pod \"prometheus-operator-68bcd7f987-7glw9\" (UID: \"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:34.681195 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.681164 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:34.708412 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:34.708386 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" Sep 11 16:18:35.022436 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:35.022201 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-7glw9"] Sep 11 16:18:35.345120 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:35.345014 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:35.422633 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:35.422587 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:35.527986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:35.527945 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:35.628959 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:35.628879 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:36.127778 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:36.127738 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:36.234926 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:36.234896 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:36.240916 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:36.240885 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:36.375036 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:36.374999 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2edf7d_7c11_4d5f_9ee5_4a7157bc7faa.slice/crio-3792e6dfcf6ba92a7aa99be081e1fece30d1fcc8f42c009f37929f9483d2cca1 WatchSource:0}: Error finding container 3792e6dfcf6ba92a7aa99be081e1fece30d1fcc8f42c009f37929f9483d2cca1: Status 404 returned error can't find the container with id 3792e6dfcf6ba92a7aa99be081e1fece30d1fcc8f42c009f37929f9483d2cca1 Sep 11 16:18:36.411002 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:36.410969 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" event={"ID":"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa","Type":"ContainerStarted","Data":"3792e6dfcf6ba92a7aa99be081e1fece30d1fcc8f42c009f37929f9483d2cca1"} Sep 11 16:18:36.980277 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:36.980244 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:37.139312 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:37.139265 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:39.169421 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:39.169384 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:39.305203 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:39.305162 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:40.844457 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:40.844362 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:42.547235 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:42.547194 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:42.626596 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:42.626539 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:44.357087 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:44.357058 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:44.456141 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:44.456098 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:45.785759 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:45.785623 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:45.879571 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:45.879538 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:46.305396 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.305360 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cfc5f8858-r58xm"] Sep 11 16:18:46.448759 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.448725 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-j7pkn" event={"ID":"172fbab1-39a1-4a58-bdab-94faf02f1526","Type":"ContainerStarted","Data":"e44ace5bf6572c163137c1341edf19dd19fe0976414f2921bd775ea34a5baefd"} Sep 11 16:18:46.448920 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.448827 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:18:46.450477 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.450457 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:46.450582 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.450482 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfc5f8858-r58xm" event={"ID":"665ee9f0-e0ed-4824-a479-bd366d312452","Type":"ContainerStarted","Data":"cba032167d8b86b05a678f4571ac88e21e791d63b8f4c3cf645aead4fa9ea0ac"} Sep 11 16:18:46.497422 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.497353 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-j7pkn" podStartSLOduration=69.580287862 podStartE2EDuration="1m23.497329006s" totalImagesPullingTime="13.917041144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:46.4710577 +0000 UTC m=+82.886155243" watchObservedRunningTime="2026-09-11 16:18:46.497329006 +0000 UTC m=+82.912426551" Sep 11 16:18:46.560960 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.560870 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:46.652479 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:46.652448 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:47.449998 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.449963 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:47.456719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.456383 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" event={"ID":"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa","Type":"ContainerStarted","Data":"8187b72924e80cd412f53b50cb88abce21b9a221378d035aefbcfa82ca135aa5"} Sep 11 16:18:47.456719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.456501 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" event={"ID":"2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa","Type":"ContainerStarted","Data":"fb6af2a44bca80f6f6731519f9168d551480ae4ad7e7221969b797aca35772b3"} Sep 11 16:18:47.458317 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.458285 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-7mr9t" event={"ID":"9e992fc4-f472-48bc-b9fa-bccf95275e45","Type":"ContainerStarted","Data":"3aedcaa21b3fbf54830689c8324a58e4aa424d7363ec0030ac38d6de5d7335c8"} Sep 11 16:18:47.458458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.458335 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:47.460335 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.460304 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xrwjl" event={"ID":"b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09","Type":"ContainerStarted","Data":"ede9cda4d4365172eec75262162da09f0314d05cd7a9a0c9a8eb1442078c1cf0"} Sep 11 16:18:47.466045 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.466023 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-7mr9t" Sep 11 16:18:47.476329 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.476280 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cfc5f8858-r58xm" podStartSLOduration=4.521223074 podStartE2EDuration="18.476259011s" totalImagesPullingTime="13.955035937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:46.500754803 +0000 UTC m=+82.915852347" watchObservedRunningTime="2026-09-11 16:18:47.476259011 +0000 UTC m=+83.891356555" Sep 11 16:18:47.477133 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.477088 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-7glw9" podStartSLOduration=2.742347625 podStartE2EDuration="13.47707389s" totalImagesPullingTime="10.734726265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:47.475811379 +0000 UTC m=+83.890908922" watchObservedRunningTime="2026-09-11 16:18:47.47707389 +0000 UTC m=+83.892171426" Sep 11 16:18:47.495095 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:47.495032 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xrwjl" podStartSLOduration=70.522340729 podStartE2EDuration="1m24.495012561s" totalImagesPullingTime="13.972671832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:47.49400289 +0000 UTC m=+83.909100437" watchObservedRunningTime="2026-09-11 16:18:47.495012561 +0000 UTC m=+83.910110104" Sep 11 16:18:48.292244 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:48.292204 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:48.392045 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:48.392007 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:48.464664 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:48.464624 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:49.551444 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.551407 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:49.680255 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.680226 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:18:49.775368 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.775331 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:49.831529 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.831433 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-7mr9t" podStartSLOduration=3.713737707 podStartE2EDuration="23.831413704s" totalImagesPullingTime="20.117675997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:47.515849358 +0000 UTC m=+83.930946902" watchObservedRunningTime="2026-09-11 16:18:49.831413704 +0000 UTC m=+86.246511247" Sep 11 16:18:49.832274 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.832252 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-c994r"] Sep 11 16:18:49.856807 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.856782 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hjg5p"] Sep 11 16:18:49.856940 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.856923 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:49.859781 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.859754 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-k88b4\"" Sep 11 16:18:49.859991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.859970 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 11 16:18:49.861033 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.861012 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 11 16:18:49.861140 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.861078 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 11 16:18:49.877374 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.877352 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-c994r"] Sep 11 16:18:49.877484 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.877468 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:49.882813 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.882794 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 11 16:18:49.882930 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.882890 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 11 16:18:49.883095 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.883009 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wc8kf\"" Sep 11 16:18:49.885921 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.885903 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 11 16:18:49.893843 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.893817 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:49.934486 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.934453 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:49.934633 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.934519 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:49.934633 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.934582 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:49.934769 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.934666 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxnmm\" (UniqueName: \"kubernetes.io/projected/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-api-access-lxnmm\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:49.934769 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.934722 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:49.934769 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:49.934750 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.035811 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.035769 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.035991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.035828 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-root\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.035991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.035870 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxnmm\" (UniqueName: \"kubernetes.io/projected/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-api-access-lxnmm\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.035991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.035898 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-accelerators-collector-config\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.035991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.035931 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.036666 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036635 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb941391-2259-4a87-8a26-e679bc8ef60f-metrics-client-ca\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036701 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.036806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036726 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036760 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-wtmp\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036777 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-textfile\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036805 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.036996 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036832 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-sys\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036996 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036872 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.036996 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036894 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-tls\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036996 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr4k9\" (UniqueName: \"kubernetes.io/projected/cb941391-2259-4a87-8a26-e679bc8ef60f-kube-api-access-nr4k9\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.036996 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036207 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.037195 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.036585 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.037195 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:50.037078 2553 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 11 16:18:50.037195 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:50.037131 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-tls podName:43c3f62e-6fd8-4bd2-8082-8750a79ec05c nodeName:}" failed. No retries permitted until 2026-09-11 16:18:50.537113331 +0000 UTC m=+86.952210865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-c994r" (UID: "43c3f62e-6fd8-4bd2-8082-8750a79ec05c") : secret "kube-state-metrics-tls" not found Sep 11 16:18:50.038248 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.038207 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.040127 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.040105 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.044792 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.044766 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxnmm\" (UniqueName: \"kubernetes.io/projected/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-api-access-lxnmm\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.118605 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.118525 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:50.137665 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137638 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-wtmp\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.137842 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137674 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-textfile\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.137842 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137734 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-sys\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.137842 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137769 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-tls\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.137842 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137797 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr4k9\" (UniqueName: \"kubernetes.io/projected/cb941391-2259-4a87-8a26-e679bc8ef60f-kube-api-access-nr4k9\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138047 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137840 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-wtmp\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138047 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137847 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-root\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138047 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137894 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-root\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138047 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137951 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-accelerators-collector-config\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138047 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.137988 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb941391-2259-4a87-8a26-e679bc8ef60f-metrics-client-ca\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138047 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.138034 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138323 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.138215 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-textfile\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138323 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.138274 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb941391-2259-4a87-8a26-e679bc8ef60f-sys\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.138418 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:50.138352 2553 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 11 16:18:50.138471 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:50.138422 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-tls podName:cb941391-2259-4a87-8a26-e679bc8ef60f nodeName:}" failed. No retries permitted until 2026-09-11 16:18:50.63840287 +0000 UTC m=+87.053500390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-tls") pod "node-exporter-hjg5p" (UID: "cb941391-2259-4a87-8a26-e679bc8ef60f") : secret "node-exporter-tls" not found Sep 11 16:18:50.139027 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.138999 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-accelerators-collector-config\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.139182 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.139153 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb941391-2259-4a87-8a26-e679bc8ef60f-metrics-client-ca\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.141266 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.141244 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.147359 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.147337 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr4k9\" (UniqueName: \"kubernetes.io/projected/cb941391-2259-4a87-8a26-e679bc8ef60f-kube-api-access-nr4k9\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.217309 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.217274 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:50.315547 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.315502 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:50.372809 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.372726 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:50.540947 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.540912 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.543930 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.543904 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43c3f62e-6fd8-4bd2-8082-8750a79ec05c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-c994r\" (UID: \"43c3f62e-6fd8-4bd2-8082-8750a79ec05c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.550471 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.550437 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:50.642459 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.642371 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-tls\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.645250 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.645223 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb941391-2259-4a87-8a26-e679bc8ef60f-node-exporter-tls\") pod \"node-exporter-hjg5p\" (UID: \"cb941391-2259-4a87-8a26-e679bc8ef60f\") " pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.769738 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.769702 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" Sep 11 16:18:50.777719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.777670 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:50.788669 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.788635 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hjg5p" Sep 11 16:18:50.878315 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:50.878279 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:51.081107 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.081070 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-c994r"] Sep 11 16:18:51.087419 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:51.087388 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c3f62e_6fd8_4bd2_8082_8750a79ec05c.slice/crio-ef84df0097b5130029e8f868dce56abe7d9da45466f24436ee48c36a72ca7033 WatchSource:0}: Error finding container ef84df0097b5130029e8f868dce56abe7d9da45466f24436ee48c36a72ca7033: Status 404 returned error can't find the container with id ef84df0097b5130029e8f868dce56abe7d9da45466f24436ee48c36a72ca7033 Sep 11 16:18:51.097231 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.097202 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:51.248230 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.248183 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" podUID="ebe454a6-746f-4012-a5a4-adbb251afcc4" containerName="registry" containerID="cri-o://1bd8d35bbe478909d426d635386221e034b9cc74baa33d283400ec096a3260af" gracePeriod=30 Sep 11 16:18:51.482719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.482027 2553 generic.go:356] "Generic (PLEG): container finished" podID="ebe454a6-746f-4012-a5a4-adbb251afcc4" containerID="1bd8d35bbe478909d426d635386221e034b9cc74baa33d283400ec096a3260af" exitCode=0 Sep 11 16:18:51.482719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.482120 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" event={"ID":"ebe454a6-746f-4012-a5a4-adbb251afcc4","Type":"ContainerDied","Data":"1bd8d35bbe478909d426d635386221e034b9cc74baa33d283400ec096a3260af"} Sep 11 16:18:51.487761 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.484124 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjg5p" event={"ID":"cb941391-2259-4a87-8a26-e679bc8ef60f","Type":"ContainerStarted","Data":"26f615b6c6e27a71af6078f4f3fbd8ffae807c44d5c46b824ce28d8f971b0729"} Sep 11 16:18:51.487761 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.486219 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" event={"ID":"43c3f62e-6fd8-4bd2-8082-8750a79ec05c","Type":"ContainerStarted","Data":"ef84df0097b5130029e8f868dce56abe7d9da45466f24436ee48c36a72ca7033"} Sep 11 16:18:51.566885 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.566831 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:51.652500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652407 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-image-registry-private-configuration\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.652500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652469 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-certificates\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.653038 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652506 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.653038 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652543 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebe454a6-746f-4012-a5a4-adbb251afcc4-ca-trust-extracted\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.653038 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652587 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-trusted-ca\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.653038 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652628 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjxt8\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-kube-api-access-rjxt8\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.653038 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.652654 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-bound-sa-token\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.654257 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.654164 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:18:51.654436 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.654392 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:18:51.655572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.655544 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-installation-pull-secrets\") pod \"ebe454a6-746f-4012-a5a4-adbb251afcc4\" (UID: \"ebe454a6-746f-4012-a5a4-adbb251afcc4\") " Sep 11 16:18:51.655873 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.655851 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-trusted-ca\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.655950 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.655879 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-certificates\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.657961 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.657616 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:18:51.658082 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.658012 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-kube-api-access-rjxt8" (OuterVolumeSpecName: "kube-api-access-rjxt8") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "kube-api-access-rjxt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 16:18:51.658671 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.658578 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 16:18:51.658795 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.658756 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:18:51.660269 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.660212 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 16:18:51.667381 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.666659 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe454a6-746f-4012-a5a4-adbb251afcc4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ebe454a6-746f-4012-a5a4-adbb251afcc4" (UID: "ebe454a6-746f-4012-a5a4-adbb251afcc4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 16:18:51.756576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.756541 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-image-registry-private-configuration\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.756576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.756573 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-registry-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.756576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.756584 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebe454a6-746f-4012-a5a4-adbb251afcc4-ca-trust-extracted\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.756876 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.756593 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjxt8\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-kube-api-access-rjxt8\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.756876 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.756607 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebe454a6-746f-4012-a5a4-adbb251afcc4-bound-sa-token\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:51.756876 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:51.756620 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebe454a6-746f-4012-a5a4-adbb251afcc4-installation-pull-secrets\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:18:52.492144 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.492112 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" Sep 11 16:18:52.492144 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.492131 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5dccd5bf64-gb52d" event={"ID":"ebe454a6-746f-4012-a5a4-adbb251afcc4","Type":"ContainerDied","Data":"df116542c98931fe06b4066eed7f83ed1e86055b1012a37f1decb668ff83569c"} Sep 11 16:18:52.492396 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.492218 2553 scope.go:117] "RemoveContainer" containerID="1bd8d35bbe478909d426d635386221e034b9cc74baa33d283400ec096a3260af" Sep 11 16:18:52.512908 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.512878 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5dccd5bf64-gb52d"] Sep 11 16:18:52.516843 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.516811 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5dccd5bf64-gb52d"] Sep 11 16:18:52.732310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.732264 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:52.775205 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.775125 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7"] Sep 11 16:18:52.775487 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.775461 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe454a6-746f-4012-a5a4-adbb251afcc4" containerName="registry" Sep 11 16:18:52.775487 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.775484 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe454a6-746f-4012-a5a4-adbb251afcc4" containerName="registry" Sep 11 16:18:52.775701 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.775570 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe454a6-746f-4012-a5a4-adbb251afcc4" containerName="registry" Sep 11 16:18:52.794981 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.794948 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.802814 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.799351 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7"] Sep 11 16:18:52.803161 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.803136 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 11 16:18:52.803475 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.803455 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 11 16:18:52.803641 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.803622 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d5c468oh8l816\"" Sep 11 16:18:52.803769 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.803748 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 11 16:18:52.803830 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.803817 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kckdw\"" Sep 11 16:18:52.806653 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.806489 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 11 16:18:52.806901 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.806554 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 11 16:18:52.832071 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.832028 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:52.967073 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.967042 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:52.968144 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968048 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968144 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968090 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968347 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-tls\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968347 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968242 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968347 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968278 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-metrics-client-ca\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968347 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968325 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll7hv\" (UniqueName: \"kubernetes.io/projected/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-kube-api-access-ll7hv\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968561 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968358 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:52.968561 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:52.968408 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-grpc-tls\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.068859 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068823 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.068999 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068866 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-tls\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.068999 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068901 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.068999 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068925 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-metrics-client-ca\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.068999 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068951 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll7hv\" (UniqueName: \"kubernetes.io/projected/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-kube-api-access-ll7hv\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.068999 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068981 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.069249 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.069030 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-grpc-tls\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.069249 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.069099 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.070015 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.069983 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-metrics-client-ca\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.070117 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.068822 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:53.079768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.073338 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.079768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.073462 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-grpc-tls\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.079768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.073592 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.079768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.074457 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-tls\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.079768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.075636 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.079768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.079082 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll7hv\" (UniqueName: \"kubernetes.io/projected/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-kube-api-access-ll7hv\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.080308 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.080285 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-ccddbcc9c-dsvp7\" (UID: \"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b\") " pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.114385 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.114355 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:18:53.303271 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.303246 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:53.444859 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.444825 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7"] Sep 11 16:18:53.451185 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:53.451148 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c6f21f_5b5f_470d_90ea_cdd5aa27bd9b.slice/crio-03bcf54584630f92d326f6522423b9a949061f6d3ed8aff11f68a63313f2a88d WatchSource:0}: Error finding container 03bcf54584630f92d326f6522423b9a949061f6d3ed8aff11f68a63313f2a88d: Status 404 returned error can't find the container with id 03bcf54584630f92d326f6522423b9a949061f6d3ed8aff11f68a63313f2a88d Sep 11 16:18:53.500627 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.500593 2553 generic.go:356] "Generic (PLEG): container finished" podID="cb941391-2259-4a87-8a26-e679bc8ef60f" containerID="647a3ca2eec19a0e04f5a538220d5c3917c71a1de0bf2971f886d55b010e56b5" exitCode=0 Sep 11 16:18:53.500786 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.500733 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjg5p" event={"ID":"cb941391-2259-4a87-8a26-e679bc8ef60f","Type":"ContainerDied","Data":"647a3ca2eec19a0e04f5a538220d5c3917c71a1de0bf2971f886d55b010e56b5"} Sep 11 16:18:53.503254 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.503232 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" event={"ID":"43c3f62e-6fd8-4bd2-8082-8750a79ec05c","Type":"ContainerStarted","Data":"5fcc0d406fcb6e56d761b02d309cb25eaa387d70fa034d475fae4b6ae2f892c9"} Sep 11 16:18:53.503340 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.503286 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" event={"ID":"43c3f62e-6fd8-4bd2-8082-8750a79ec05c","Type":"ContainerStarted","Data":"57a962c66af35dda5438db2211b0504fca5b3997b2b41a21c07e4e9998bfcf85"} Sep 11 16:18:53.505773 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.505742 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"03bcf54584630f92d326f6522423b9a949061f6d3ed8aff11f68a63313f2a88d"} Sep 11 16:18:53.563167 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:53.563118 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:54.081273 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.081237 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe454a6-746f-4012-a5a4-adbb251afcc4" path="/var/lib/kubelet/pods/ebe454a6-746f-4012-a5a4-adbb251afcc4/volumes" Sep 11 16:18:54.518866 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.518115 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjg5p" event={"ID":"cb941391-2259-4a87-8a26-e679bc8ef60f","Type":"ContainerStarted","Data":"83857ab9a4acfa90d9d7eb0444192c0cce2318ef19e4491a678cd815328dcb08"} Sep 11 16:18:54.518866 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.518176 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjg5p" event={"ID":"cb941391-2259-4a87-8a26-e679bc8ef60f","Type":"ContainerStarted","Data":"ab832b3263f37f164f57bb7b71a982bac6419de00e507a2160f831a4295faf78"} Sep 11 16:18:54.522836 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.522801 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" event={"ID":"43c3f62e-6fd8-4bd2-8082-8750a79ec05c","Type":"ContainerStarted","Data":"1a3678de5721ff18f02de0c1d9166d36abbcdb3ef7257e5ba47cc0485ecf90fb"} Sep 11 16:18:54.563590 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.563542 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv"] Sep 11 16:18:54.573598 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.573538 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hjg5p" podStartSLOduration=3.309388936 podStartE2EDuration="5.573515314s" totalImagesPullingTime="2.158887242s" totalInitContainerRuntime="105.239136ms" isStatefulPod=true podCreationTimestamp="2026-09-11 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:54.570354117 +0000 UTC m=+90.985451661" watchObservedRunningTime="2026-09-11 16:18:54.573515314 +0000 UTC m=+90.988612859" Sep 11 16:18:54.580614 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.580587 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76b7fb4b7-4ntrq"] Sep 11 16:18:54.581906 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.580565 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:54.587753 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.587729 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 11 16:18:54.590746 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.590724 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4ztjd\"" Sep 11 16:18:54.591306 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.591264 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:54.598552 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.598511 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv"] Sep 11 16:18:54.598552 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.598549 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b7fb4b7-4ntrq"] Sep 11 16:18:54.598746 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.598637 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.606989 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.606960 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 11 16:18:54.613673 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.613574 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-c994r" podStartSLOduration=3.43293125 podStartE2EDuration="5.613558011s" totalImagesPullingTime="2.180626761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:54.613295239 +0000 UTC m=+91.028392783" watchObservedRunningTime="2026-09-11 16:18:54.613558011 +0000 UTC m=+91.028655607" Sep 11 16:18:54.684230 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684182 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-service-ca\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.684659 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684509 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-config\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.684659 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684615 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-oauth-serving-cert\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.684846 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684709 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-trusted-ca-bundle\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.684846 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684810 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mspgx\" (UniqueName: \"kubernetes.io/projected/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-kube-api-access-mspgx\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.684846 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c80e75b2-05fd-4328-b748-4bdeca6242c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:54.685000 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684869 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zssqt\" (UniqueName: \"kubernetes.io/projected/c80e75b2-05fd-4328-b748-4bdeca6242c7-kube-api-access-zssqt\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:54.685000 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684898 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-serving-cert\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.685000 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.684947 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-oauth-config\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.693195 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.693166 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:54.785721 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785623 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-service-ca\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.785721 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785665 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-config\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.785721 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-oauth-serving-cert\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.785983 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785739 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-trusted-ca-bundle\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.785983 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785787 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mspgx\" (UniqueName: \"kubernetes.io/projected/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-kube-api-access-mspgx\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.785983 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785815 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c80e75b2-05fd-4328-b748-4bdeca6242c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:54.785983 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785841 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zssqt\" (UniqueName: \"kubernetes.io/projected/c80e75b2-05fd-4328-b748-4bdeca6242c7-kube-api-access-zssqt\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:54.785983 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785862 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-serving-cert\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.785983 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.785892 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-oauth-config\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.786712 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:54.786321 2553 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 11 16:18:54.786712 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:18:54.786447 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e75b2-05fd-4328-b748-4bdeca6242c7-monitoring-plugin-cert podName:c80e75b2-05fd-4328-b748-4bdeca6242c7 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:55.286424852 +0000 UTC m=+91.701522383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c80e75b2-05fd-4328-b748-4bdeca6242c7-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-vp8hv" (UID: "c80e75b2-05fd-4328-b748-4bdeca6242c7") : secret "monitoring-plugin-cert" not found Sep 11 16:18:54.786712 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.786537 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-config\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.787168 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.787145 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-service-ca\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.787454 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.787427 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-oauth-serving-cert\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.787716 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.787670 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-trusted-ca-bundle\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.788927 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.788903 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-oauth-config\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.789491 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.789465 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-serving-cert\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.797176 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.797154 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zssqt\" (UniqueName: \"kubernetes.io/projected/c80e75b2-05fd-4328-b748-4bdeca6242c7-kube-api-access-zssqt\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:54.797786 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.797763 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mspgx\" (UniqueName: \"kubernetes.io/projected/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-kube-api-access-mspgx\") pod \"console-76b7fb4b7-4ntrq\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.799011 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.798836 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:54.915790 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.915751 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:18:54.977988 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:54.977946 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:55.131194 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.131070 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:55.233386 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.233305 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:55.290594 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.290552 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c80e75b2-05fd-4328-b748-4bdeca6242c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:55.293639 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.293610 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c80e75b2-05fd-4328-b748-4bdeca6242c7-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-vp8hv\" (UID: \"c80e75b2-05fd-4328-b748-4bdeca6242c7\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:55.502183 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.502151 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:55.784114 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.784012 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76b7fb4b7-4ntrq"] Sep 11 16:18:55.812400 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:55.812363 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv"] Sep 11 16:18:55.928074 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:55.928030 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3fe7ba0_1151_444e_8e6a_dcbe1118965f.slice/crio-2636b6fd50cc98ca25b471064e6a3d70534b4451177b38322dc00abec796957e WatchSource:0}: Error finding container 2636b6fd50cc98ca25b471064e6a3d70534b4451177b38322dc00abec796957e: Status 404 returned error can't find the container with id 2636b6fd50cc98ca25b471064e6a3d70534b4451177b38322dc00abec796957e Sep 11 16:18:55.929133 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:55.929113 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80e75b2_05fd_4328_b748_4bdeca6242c7.slice/crio-c064d709a03652520dca9959dd2eb1f89209fda951150a50523389d05ad7b331 WatchSource:0}: Error finding container c064d709a03652520dca9959dd2eb1f89209fda951150a50523389d05ad7b331: Status 404 returned error can't find the container with id c064d709a03652520dca9959dd2eb1f89209fda951150a50523389d05ad7b331 Sep 11 16:18:56.123883 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.123849 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:18:56.126367 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.126341 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 16:18:56.141733 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.141706 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.145667 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.145635 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 11 16:18:56.145810 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.145649 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 11 16:18:56.146362 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.146053 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 11 16:18:56.147357 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147285 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 11 16:18:56.147357 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147351 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 11 16:18:56.147512 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147386 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-dlmnmnkuf8bgk\"" Sep 11 16:18:56.147563 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147530 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 11 16:18:56.148101 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147627 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 11 16:18:56.148101 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147667 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 11 16:18:56.148101 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147810 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 11 16:18:56.148101 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.147967 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 11 16:18:56.148101 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.148001 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j4fd5\"" Sep 11 16:18:56.150191 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.150175 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 11 16:18:56.150463 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.150441 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 11 16:18:56.154191 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.154120 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 11 16:18:56.162969 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.162527 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 16:18:56.167457 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.164708 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 11 16:18:56.168364 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.168200 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 11 16:18:56.299990 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.299942 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqr8\" (UniqueName: \"kubernetes.io/projected/abc53d86-47e4-488e-a617-2205414a2a40-kube-api-access-5zqr8\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300136 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300032 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300136 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300057 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300136 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300099 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300161 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300179 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300194 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-web-config\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300216 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300258 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-config\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300279 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300304 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300321 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300355 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300395 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300427 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300445 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300476 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/abc53d86-47e4-488e-a617-2205414a2a40-config-out\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300505 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300572 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/abc53d86-47e4-488e-a617-2205414a2a40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.300651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.300602 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.401787 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.401751 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.401948 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.401796 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.401948 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.401813 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.401948 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.401896 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-web-config\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402332 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402306 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402444 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-config\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402444 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402405 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402445 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402469 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402490 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402514 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402550 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402611 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402634 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402669 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/abc53d86-47e4-488e-a617-2205414a2a40-config-out\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402742 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402777 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/abc53d86-47e4-488e-a617-2205414a2a40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.402829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402799 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.403106 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402842 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqr8\" (UniqueName: \"kubernetes.io/projected/abc53d86-47e4-488e-a617-2205414a2a40-kube-api-access-5zqr8\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.403106 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402878 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.403106 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.402907 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.403443 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.403414 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.403765 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.403738 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.405785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.405755 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.406469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.406445 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.406548 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.406475 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.410554 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.409604 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.410554 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.409642 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/abc53d86-47e4-488e-a617-2205414a2a40-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.410554 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.410494 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.415735 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.415370 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.416581 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.415652 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.416581 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.416364 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.416581 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.416476 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-web-config\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.418219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.416731 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-config\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.418219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.417037 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/abc53d86-47e4-488e-a617-2205414a2a40-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.418219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.417493 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.418219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.417599 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.418433 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.418327 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/abc53d86-47e4-488e-a617-2205414a2a40-config-out\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.419634 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.419594 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqr8\" (UniqueName: \"kubernetes.io/projected/abc53d86-47e4-488e-a617-2205414a2a40-kube-api-access-5zqr8\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.420037 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.419960 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/abc53d86-47e4-488e-a617-2205414a2a40-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"abc53d86-47e4-488e-a617-2205414a2a40\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.472723 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.472600 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:18:56.536303 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.536156 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"9eb50163e981a01d9d09f9c33bc838f98a69957f6ca5cefb50d62636feb5a7d2"} Sep 11 16:18:56.536303 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.536233 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"279314468e528f494c75b0ec983558bee30472159dfa6b0e76ff03cdeb3388f2"} Sep 11 16:18:56.536303 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.536269 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"21df95509c0a9c244e5b658e4c08ced26182abbdeaae9976077260c569e1323b"} Sep 11 16:18:56.538138 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.538057 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" event={"ID":"c80e75b2-05fd-4328-b748-4bdeca6242c7","Type":"ContainerStarted","Data":"c064d709a03652520dca9959dd2eb1f89209fda951150a50523389d05ad7b331"} Sep 11 16:18:56.541108 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.541021 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7fb4b7-4ntrq" event={"ID":"f3fe7ba0-1151-444e-8e6a-dcbe1118965f","Type":"ContainerStarted","Data":"f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c"} Sep 11 16:18:56.541108 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.541072 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7fb4b7-4ntrq" event={"ID":"f3fe7ba0-1151-444e-8e6a-dcbe1118965f","Type":"ContainerStarted","Data":"2636b6fd50cc98ca25b471064e6a3d70534b4451177b38322dc00abec796957e"} Sep 11 16:18:56.563583 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.563533 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76b7fb4b7-4ntrq" podStartSLOduration=2.563516661 podStartE2EDuration="2.563516661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:56.562193865 +0000 UTC m=+92.977291409" watchObservedRunningTime="2026-09-11 16:18:56.563516661 +0000 UTC m=+92.978614204" Sep 11 16:18:56.809625 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:56.809543 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 11 16:18:56.815574 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:18:56.815540 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc53d86_47e4_488e_a617_2205414a2a40.slice/crio-3e313c31ae3e28e47d23a0086dbb58664b69e4f02f5b5c9f29017a98c9546c69 WatchSource:0}: Error finding container 3e313c31ae3e28e47d23a0086dbb58664b69e4f02f5b5c9f29017a98c9546c69: Status 404 returned error can't find the container with id 3e313c31ae3e28e47d23a0086dbb58664b69e4f02f5b5c9f29017a98c9546c69 Sep 11 16:18:57.545382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:57.545338 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"3e313c31ae3e28e47d23a0086dbb58664b69e4f02f5b5c9f29017a98c9546c69"} Sep 11 16:18:58.551544 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:58.551501 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" event={"ID":"c80e75b2-05fd-4328-b748-4bdeca6242c7","Type":"ContainerStarted","Data":"eb7fd16e5ecc10a1bf085b07e09a80fb2f88b7477733b8dce6d9b0f993fcc659"} Sep 11 16:18:58.551965 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:58.551568 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:58.554921 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:58.554895 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"2cf85ab1bb575c6835d437f94b16122112f5b9a5d45c12de3425f509e30b948a"} Sep 11 16:18:58.555033 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:58.554954 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"89623799abdf407966fe6fdc5cfb77a4f0987604be629518da43823eeab0987a"} Sep 11 16:18:58.557022 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:58.556997 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" Sep 11 16:18:58.576154 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:58.576110 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-vp8hv" podStartSLOduration=2.650505216 podStartE2EDuration="4.57609168s" totalImagesPullingTime="1.925586464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:58.575041312 +0000 UTC m=+94.990138867" watchObservedRunningTime="2026-09-11 16:18:58.57609168 +0000 UTC m=+94.991189224" Sep 11 16:18:59.560514 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:59.560486 2553 generic.go:356] "Generic (PLEG): container finished" podID="abc53d86-47e4-488e-a617-2205414a2a40" containerID="2d5eee6ca794aa1fec20bf68d7d9839d564a444629624810acd95c11b76f2936" exitCode=0 Sep 11 16:18:59.561034 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:59.560562 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerDied","Data":"2d5eee6ca794aa1fec20bf68d7d9839d564a444629624810acd95c11b76f2936"} Sep 11 16:18:59.563898 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:59.563868 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" event={"ID":"d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b","Type":"ContainerStarted","Data":"00c06df6e11d051d1a5ab1d98a3f8322b546cde9c9edd4a96d3797e4a5f0a205"} Sep 11 16:18:59.627748 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:18:59.627669 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" podStartSLOduration=3.605449758 podStartE2EDuration="7.627649808s" totalImagesPullingTime="4.02220005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:59.625878436 +0000 UTC m=+96.040975980" watchObservedRunningTime="2026-09-11 16:18:59.627649808 +0000 UTC m=+96.042747385" Sep 11 16:19:00.568512 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:00.568404 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:19:01.579034 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:01.579006 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-ccddbcc9c-dsvp7" Sep 11 16:19:03.586567 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:03.586486 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"f6428419137610deb93137369b965bff151a382fe1998d8908fc42e6c451e89f"} Sep 11 16:19:04.594393 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.594355 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"4384eb851e1713cbc4e218dbeaffbcf3c7a6e59c8054447750a35fa2d195a626"} Sep 11 16:19:04.594874 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.594436 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"1920621432f344e99a6d193a6e35d72951ff75c5680288a4257889a736bde273"} Sep 11 16:19:04.594874 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.594478 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"7e0ad6ad6d7b42a3de6f1292547d12aabdf945966bf7359e482d0377e9f342e6"} Sep 11 16:19:04.594874 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.594513 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"6aa036cd4301fb2cdfaea24129596eb1c28d3c940b9ab32b8962bf45403c537d"} Sep 11 16:19:04.594874 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.594548 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"abc53d86-47e4-488e-a617-2205414a2a40","Type":"ContainerStarted","Data":"93f633a2a1cac60ae697c671e5371537e448c71e4c5f8644b11f0a843376c99a"} Sep 11 16:19:04.629882 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.629834 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.879832939 podStartE2EDuration="8.629816996s" totalImagesPullingTime="5.731164563s" totalInitContainerRuntime="18.819494ms" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:19:04.627887839 +0000 UTC m=+101.042985381" watchObservedRunningTime="2026-09-11 16:19:04.629816996 +0000 UTC m=+101.044914540" Sep 11 16:19:04.916383 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.916297 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:19:04.916535 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.916389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:19:04.921269 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:04.921232 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:19:05.603039 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:05.603010 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:19:05.775805 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:05.775774 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:05.892746 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:05.892651 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:05.998008 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:05.997961 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:06.472828 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:06.472783 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:19:09.139217 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:09.139179 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:09.225541 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:09.225501 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:11.066661 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.066628 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:11.166482 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.166453 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:11.231263 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.231234 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:11.472042 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.471997 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cfc5f8858-r58xm" podUID="665ee9f0-e0ed-4824-a479-bd366d312452" containerName="console" containerID="cri-o://cba032167d8b86b05a678f4571ac88e21e791d63b8f4c3cf645aead4fa9ea0ac" gracePeriod=15 Sep 11 16:19:11.629757 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.629722 2553 generic.go:356] "Generic (PLEG): container finished" podID="665ee9f0-e0ed-4824-a479-bd366d312452" containerID="cba032167d8b86b05a678f4571ac88e21e791d63b8f4c3cf645aead4fa9ea0ac" exitCode=0 Sep 11 16:19:11.629935 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.629793 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfc5f8858-r58xm" event={"ID":"665ee9f0-e0ed-4824-a479-bd366d312452","Type":"ContainerDied","Data":"cba032167d8b86b05a678f4571ac88e21e791d63b8f4c3cf645aead4fa9ea0ac"} Sep 11 16:19:11.740982 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.740906 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:19:11.840537 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840501 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-service-ca\") pod \"665ee9f0-e0ed-4824-a479-bd366d312452\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " Sep 11 16:19:11.840774 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840551 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-oauth-serving-cert\") pod \"665ee9f0-e0ed-4824-a479-bd366d312452\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " Sep 11 16:19:11.840774 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840567 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29swq\" (UniqueName: \"kubernetes.io/projected/665ee9f0-e0ed-4824-a479-bd366d312452-kube-api-access-29swq\") pod \"665ee9f0-e0ed-4824-a479-bd366d312452\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " Sep 11 16:19:11.840774 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840611 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-serving-cert\") pod \"665ee9f0-e0ed-4824-a479-bd366d312452\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " Sep 11 16:19:11.840774 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840637 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-console-config\") pod \"665ee9f0-e0ed-4824-a479-bd366d312452\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " Sep 11 16:19:11.840986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840836 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-oauth-config\") pod \"665ee9f0-e0ed-4824-a479-bd366d312452\" (UID: \"665ee9f0-e0ed-4824-a479-bd366d312452\") " Sep 11 16:19:11.841045 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.840999 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-service-ca" (OuterVolumeSpecName: "service-ca") pod "665ee9f0-e0ed-4824-a479-bd366d312452" (UID: "665ee9f0-e0ed-4824-a479-bd366d312452"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:19:11.841045 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.841009 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "665ee9f0-e0ed-4824-a479-bd366d312452" (UID: "665ee9f0-e0ed-4824-a479-bd366d312452"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:19:11.841143 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.841081 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-console-config" (OuterVolumeSpecName: "console-config") pod "665ee9f0-e0ed-4824-a479-bd366d312452" (UID: "665ee9f0-e0ed-4824-a479-bd366d312452"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:19:11.841143 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.841131 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-oauth-serving-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:19:11.841219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.841155 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-service-ca\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:19:11.843287 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.843263 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "665ee9f0-e0ed-4824-a479-bd366d312452" (UID: "665ee9f0-e0ed-4824-a479-bd366d312452"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:19:11.843416 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.843324 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "665ee9f0-e0ed-4824-a479-bd366d312452" (UID: "665ee9f0-e0ed-4824-a479-bd366d312452"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:19:11.843475 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.843431 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/665ee9f0-e0ed-4824-a479-bd366d312452-kube-api-access-29swq" (OuterVolumeSpecName: "kube-api-access-29swq") pod "665ee9f0-e0ed-4824-a479-bd366d312452" (UID: "665ee9f0-e0ed-4824-a479-bd366d312452"). InnerVolumeSpecName "kube-api-access-29swq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 16:19:11.941742 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.941667 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29swq\" (UniqueName: \"kubernetes.io/projected/665ee9f0-e0ed-4824-a479-bd366d312452-kube-api-access-29swq\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:19:11.941742 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.941743 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-serving-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:19:11.941932 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.941761 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/665ee9f0-e0ed-4824-a479-bd366d312452-console-config\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:19:11.941932 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:11.941776 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/665ee9f0-e0ed-4824-a479-bd366d312452-console-oauth-config\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:19:12.635019 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:12.634986 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cfc5f8858-r58xm" event={"ID":"665ee9f0-e0ed-4824-a479-bd366d312452","Type":"ContainerDied","Data":"4306ec61c9cfdf93cb150e482c7ed94ec6bdeb564a62f916c938586c44992d0b"} Sep 11 16:19:12.635019 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:12.635004 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cfc5f8858-r58xm" Sep 11 16:19:12.635626 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:12.635050 2553 scope.go:117] "RemoveContainer" containerID="cba032167d8b86b05a678f4571ac88e21e791d63b8f4c3cf645aead4fa9ea0ac" Sep 11 16:19:12.654152 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:12.654127 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cfc5f8858-r58xm"] Sep 11 16:19:12.663793 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:12.663765 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cfc5f8858-r58xm"] Sep 11 16:19:14.076381 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:14.076348 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="665ee9f0-e0ed-4824-a479-bd366d312452" path="/var/lib/kubelet/pods/665ee9f0-e0ed-4824-a479-bd366d312452/volumes" Sep 11 16:19:17.462354 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:17.462323 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-j7pkn" Sep 11 16:19:24.125668 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:24.125620 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:24.188195 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:24.188116 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:19:56.473266 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:56.473228 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:19:56.492884 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:56.492857 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:19:56.817029 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:19:56.816951 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 11 16:20:09.076165 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:09.076114 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:09.280746 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:09.280699 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:18.187870 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:18.187779 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:18.290556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:18.290523 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:18.390044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:18.390005 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:18.919862 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:18.919826 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:18.985756 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:18.985725 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:24.327925 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:24.327887 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:24.402037 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:24.402001 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:36.297622 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:36.297585 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:36.357038 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:36.357004 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:39.134281 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:39.134234 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:39.235273 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:39.235240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:39.695836 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:39.695804 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76b7fb4b7-4ntrq"] Sep 11 16:20:39.827402 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:39.827359 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:39.926832 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:39.926797 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:39.985242 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:39.985152 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:42.392040 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:42.392003 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:20:42.492090 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:20:42.492050 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:21:04.715851 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:04.715786 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76b7fb4b7-4ntrq" podUID="f3fe7ba0-1151-444e-8e6a-dcbe1118965f" containerName="console" containerID="cri-o://f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c" gracePeriod=15 Sep 11 16:21:04.956214 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:04.956193 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:21:05.051915 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.051831 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-oauth-serving-cert\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.051915 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.051862 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-serving-cert\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.051915 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.051916 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mspgx\" (UniqueName: \"kubernetes.io/projected/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-kube-api-access-mspgx\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.052174 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.051962 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-config\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.052174 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.051994 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-trusted-ca-bundle\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.052174 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.052028 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-oauth-config\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.052174 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.052050 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-service-ca\") pod \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\" (UID: \"f3fe7ba0-1151-444e-8e6a-dcbe1118965f\") " Sep 11 16:21:05.052369 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.052305 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:21:05.052422 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.052393 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-config" (OuterVolumeSpecName: "console-config") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:21:05.052499 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.052469 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:21:05.052694 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.052500 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-service-ca" (OuterVolumeSpecName: "service-ca") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:21:05.054466 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054440 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:21:05.054571 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054549 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:21:05.054571 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054560 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-kube-api-access-mspgx" (OuterVolumeSpecName: "kube-api-access-mspgx") pod "f3fe7ba0-1151-444e-8e6a-dcbe1118965f" (UID: "f3fe7ba0-1151-444e-8e6a-dcbe1118965f"). InnerVolumeSpecName "kube-api-access-mspgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 16:21:05.054816 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054795 2553 generic.go:356] "Generic (PLEG): container finished" podID="f3fe7ba0-1151-444e-8e6a-dcbe1118965f" containerID="f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c" exitCode=0 Sep 11 16:21:05.054881 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054840 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7fb4b7-4ntrq" event={"ID":"f3fe7ba0-1151-444e-8e6a-dcbe1118965f","Type":"ContainerDied","Data":"f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c"} Sep 11 16:21:05.054922 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054891 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76b7fb4b7-4ntrq" event={"ID":"f3fe7ba0-1151-444e-8e6a-dcbe1118965f","Type":"ContainerDied","Data":"2636b6fd50cc98ca25b471064e6a3d70534b4451177b38322dc00abec796957e"} Sep 11 16:21:05.054922 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054912 2553 scope.go:117] "RemoveContainer" containerID="f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c" Sep 11 16:21:05.054992 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.054848 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76b7fb4b7-4ntrq" Sep 11 16:21:05.068722 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.068703 2553 scope.go:117] "RemoveContainer" containerID="f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c" Sep 11 16:21:05.069022 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:21:05.069003 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c\": container with ID starting with f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c not found: ID does not exist" containerID="f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c" Sep 11 16:21:05.069082 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.069029 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c"} err="failed to get container status \"f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c\": rpc error: code = NotFound desc = could not find container \"f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c\": container with ID starting with f3883e536c32a89395a33f335608b8cb6f64c05021e835298fa6cb10f0ec0a6c not found: ID does not exist" Sep 11 16:21:05.080034 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.080013 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76b7fb4b7-4ntrq"] Sep 11 16:21:05.083763 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.083728 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76b7fb4b7-4ntrq"] Sep 11 16:21:05.153259 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153218 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-oauth-config\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:05.153259 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153250 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-service-ca\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:05.153259 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153265 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-oauth-serving-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:05.153500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153277 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-serving-cert\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:05.153500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153290 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mspgx\" (UniqueName: \"kubernetes.io/projected/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-kube-api-access-mspgx\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:05.153500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153303 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-console-config\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:05.153500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:05.153325 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fe7ba0-1151-444e-8e6a-dcbe1118965f-trusted-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:21:06.075778 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:06.075742 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3fe7ba0-1151-444e-8e6a-dcbe1118965f" path="/var/lib/kubelet/pods/f3fe7ba0-1151-444e-8e6a-dcbe1118965f/volumes" Sep 11 16:21:09.429443 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:09.429414 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:21:40.263869 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:21:40.263834 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:10.908085 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:10.908053 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:11.007923 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:11.007894 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:11.112174 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:11.112116 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:23.950658 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:23.950632 2553 kubelet.go:1648] "Image garbage collection succeeded" Sep 11 16:22:27.886510 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:27.886477 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:29.251722 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:29.251666 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:33.229779 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:33.229744 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:33.325841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:33.325807 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:39.336349 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:39.336309 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:39.594115 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:39.594030 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:39.744891 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:39.744860 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:40.001911 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:40.001881 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:40.297458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:40.297363 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:40.581601 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:40.581519 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:40.676534 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:40.676484 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:40.974817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:40.974782 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:41.304056 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:41.303967 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:41.572332 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:41.572249 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:41.771967 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:41.771934 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:22:41.880106 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:22:41.880032 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:31.865864 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:31.865789 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:31.963872 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:31.963843 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:33.231058 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:33.231026 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:33.332650 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:33.332615 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:44.461022 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:44.460988 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:44.654119 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:44.654086 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:44.782011 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:44.781933 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.179263 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.179228 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.300583 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.300536 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.403626 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.403592 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.504427 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.504397 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.830224 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.830137 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.901900 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.901872 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:23:58.971466 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:23:58.971418 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:06.588496 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:06.588466 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:16.508051 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:16.508022 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:16.612201 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:16.612167 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:16.712653 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:16.712622 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:16.814615 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:16.814542 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:17.698028 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:17.697996 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:17.754548 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:17.754494 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:25:39.675175 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:25:39.675140 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:26:50.310114 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:26:50.310034 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:26:50.412659 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:26:50.412605 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:01.548460 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:01.548428 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:01.745817 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:01.745789 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:01.848437 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:01.848357 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:01.922928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:01.922890 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:02.050695 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:02.050646 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:02.152896 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:02.152804 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:02.468821 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:02.468782 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:02.568829 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:02.568798 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:02.671927 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:02.671893 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:18.616148 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:18.616096 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:27:59.169979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:27:59.169946 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:10.306699 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:10.306631 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:31.490980 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.490953 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd"] Sep 11 16:28:31.491406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.491221 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="665ee9f0-e0ed-4824-a479-bd366d312452" containerName="console" Sep 11 16:28:31.491406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.491233 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="665ee9f0-e0ed-4824-a479-bd366d312452" containerName="console" Sep 11 16:28:31.491406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.491255 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fe7ba0-1151-444e-8e6a-dcbe1118965f" containerName="console" Sep 11 16:28:31.491406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.491260 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fe7ba0-1151-444e-8e6a-dcbe1118965f" containerName="console" Sep 11 16:28:31.491406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.491302 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fe7ba0-1151-444e-8e6a-dcbe1118965f" containerName="console" Sep 11 16:28:31.491406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.491309 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="665ee9f0-e0ed-4824-a479-bd366d312452" containerName="console" Sep 11 16:28:31.493055 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.493038 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:31.495807 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.495787 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 11 16:28:31.495929 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.495841 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mmq9q\"" Sep 11 16:28:31.496770 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.496754 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8975a-serving-cert\"" Sep 11 16:28:31.496826 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.496757 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8975a-kube-rbac-proxy-sar-config\"" Sep 11 16:28:31.500693 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.500659 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:31.504576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.504558 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd"] Sep 11 16:28:31.567392 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.567365 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1e63add-64d3-426b-86a5-f0c99bf23920-openshift-service-ca-bundle\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:31.567509 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.567398 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:31.668188 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.668157 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1e63add-64d3-426b-86a5-f0c99bf23920-openshift-service-ca-bundle\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:31.668188 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.668192 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:31.668368 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:28:31.668280 2553 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-8975a-serving-cert: secret "switch-graph-8975a-serving-cert" not found Sep 11 16:28:31.668368 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:28:31.668344 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls podName:b1e63add-64d3-426b-86a5-f0c99bf23920 nodeName:}" failed. No retries permitted until 2026-09-11 16:28:32.168327188 +0000 UTC m=+668.583424708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls") pod "switch-graph-8975a-67db9b666-tz2kd" (UID: "b1e63add-64d3-426b-86a5-f0c99bf23920") : secret "switch-graph-8975a-serving-cert" not found Sep 11 16:28:31.668764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.668748 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1e63add-64d3-426b-86a5-f0c99bf23920-openshift-service-ca-bundle\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:31.792913 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:31.792836 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:32.170833 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:32.170751 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:32.173201 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:32.173178 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls\") pod \"switch-graph-8975a-67db9b666-tz2kd\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:32.403173 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:32.403145 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:32.670154 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:32.670036 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd"] Sep 11 16:28:32.672670 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:28:32.672632 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e63add_64d3_426b_86a5_f0c99bf23920.slice/crio-19bedf72c568694b24e470fcb1ad8bf5abb9d527a812a00cfb46c33212a18eee WatchSource:0}: Error finding container 19bedf72c568694b24e470fcb1ad8bf5abb9d527a812a00cfb46c33212a18eee: Status 404 returned error can't find the container with id 19bedf72c568694b24e470fcb1ad8bf5abb9d527a812a00cfb46c33212a18eee Sep 11 16:28:32.674553 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:32.674539 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 16:28:32.684950 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:32.684924 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" event={"ID":"b1e63add-64d3-426b-86a5-f0c99bf23920","Type":"ContainerStarted","Data":"19bedf72c568694b24e470fcb1ad8bf5abb9d527a812a00cfb46c33212a18eee"} Sep 11 16:28:36.705375 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:36.705328 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" event={"ID":"b1e63add-64d3-426b-86a5-f0c99bf23920","Type":"ContainerStarted","Data":"19d5bf5497f219a41635c07ae03f85fcf541b5e683d4e81f0ed4b37c103dca15"} Sep 11 16:28:36.705790 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:36.705408 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:36.723051 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:36.723008 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podStartSLOduration=2.550332974 podStartE2EDuration="5.722992967s" totalImagesPullingTime="3.172659993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:28:36.722068757 +0000 UTC m=+673.137166300" watchObservedRunningTime="2026-09-11 16:28:36.722992967 +0000 UTC m=+673.138090518" Sep 11 16:28:42.714443 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:42.714411 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:28:45.692279 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:45.692221 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd"] Sep 11 16:28:45.692654 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:45.692403 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" containerID="cri-o://19d5bf5497f219a41635c07ae03f85fcf541b5e683d4e81f0ed4b37c103dca15" gracePeriod=30 Sep 11 16:28:45.736982 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:45.736953 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:45.887377 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:45.887342 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:45.996024 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:45.995977 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:46.101642 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:46.101616 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:46.212732 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:46.212691 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:46.381159 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:46.381045 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:28:47.712938 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:47.712897 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:28:52.714174 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:52.714133 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:28:57.713161 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:57.713122 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:28:57.713651 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:28:57.713242 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:29:02.712764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:02.712723 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:07.712894 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:07.712857 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:11.502358 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.502328 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx"] Sep 11 16:29:11.509282 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.509262 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:11.509480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.509452 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:11.511828 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.511810 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Sep 11 16:29:11.511929 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.511830 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Sep 11 16:29:11.514545 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.514525 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx"] Sep 11 16:29:11.595502 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.595465 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:11.595664 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.595569 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80d98814-f0dd-447f-9957-0410f03b41c0-openshift-service-ca-bundle\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:11.696254 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.696216 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80d98814-f0dd-447f-9957-0410f03b41c0-openshift-service-ca-bundle\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:11.696407 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.696287 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:11.696465 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:29:11.696445 2553 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Sep 11 16:29:11.696550 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:29:11.696527 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls podName:80d98814-f0dd-447f-9957-0410f03b41c0 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:12.196507675 +0000 UTC m=+708.611605208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls") pod "model-chainer-6f97b8c58f-jd8dx" (UID: "80d98814-f0dd-447f-9957-0410f03b41c0") : secret "model-chainer-serving-cert" not found Sep 11 16:29:11.696874 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.696856 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80d98814-f0dd-447f-9957-0410f03b41c0-openshift-service-ca-bundle\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:11.976105 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:11.976074 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:12.201328 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.201295 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:12.203960 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.203927 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls\") pod \"model-chainer-6f97b8c58f-jd8dx\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:12.419996 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.419900 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:12.688289 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.688265 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx"] Sep 11 16:29:12.690704 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:29:12.690661 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d98814_f0dd_447f_9957_0410f03b41c0.slice/crio-f62cd72da6e2489cd753a1f07ba015375396294ad181e2888c6c374ddfc13124 WatchSource:0}: Error finding container f62cd72da6e2489cd753a1f07ba015375396294ad181e2888c6c374ddfc13124: Status 404 returned error can't find the container with id f62cd72da6e2489cd753a1f07ba015375396294ad181e2888c6c374ddfc13124 Sep 11 16:29:12.713339 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.713297 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:12.843971 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.843938 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" event={"ID":"80d98814-f0dd-447f-9957-0410f03b41c0","Type":"ContainerStarted","Data":"f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0"} Sep 11 16:29:12.844149 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.843993 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" event={"ID":"80d98814-f0dd-447f-9957-0410f03b41c0","Type":"ContainerStarted","Data":"f62cd72da6e2489cd753a1f07ba015375396294ad181e2888c6c374ddfc13124"} Sep 11 16:29:12.844149 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.844025 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:12.859813 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:12.859768 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podStartSLOduration=1.859751756 podStartE2EDuration="1.859751756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:12.858893241 +0000 UTC m=+709.273990791" watchObservedRunningTime="2026-09-11 16:29:12.859751756 +0000 UTC m=+709.274849301" Sep 11 16:29:15.857072 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:15.857038 2553 generic.go:356] "Generic (PLEG): container finished" podID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerID="19d5bf5497f219a41635c07ae03f85fcf541b5e683d4e81f0ed4b37c103dca15" exitCode=0 Sep 11 16:29:15.857443 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:15.857082 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" event={"ID":"b1e63add-64d3-426b-86a5-f0c99bf23920","Type":"ContainerDied","Data":"19d5bf5497f219a41635c07ae03f85fcf541b5e683d4e81f0ed4b37c103dca15"} Sep 11 16:29:16.333692 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.333663 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:29:16.439458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.439423 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1e63add-64d3-426b-86a5-f0c99bf23920-openshift-service-ca-bundle\") pod \"b1e63add-64d3-426b-86a5-f0c99bf23920\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " Sep 11 16:29:16.439630 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.439506 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls\") pod \"b1e63add-64d3-426b-86a5-f0c99bf23920\" (UID: \"b1e63add-64d3-426b-86a5-f0c99bf23920\") " Sep 11 16:29:16.439869 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.439847 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1e63add-64d3-426b-86a5-f0c99bf23920-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b1e63add-64d3-426b-86a5-f0c99bf23920" (UID: "b1e63add-64d3-426b-86a5-f0c99bf23920"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:29:16.441886 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.441866 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b1e63add-64d3-426b-86a5-f0c99bf23920" (UID: "b1e63add-64d3-426b-86a5-f0c99bf23920"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:29:16.540538 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.540447 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1e63add-64d3-426b-86a5-f0c99bf23920-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:29:16.540538 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.540477 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b1e63add-64d3-426b-86a5-f0c99bf23920-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:29:16.861855 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.861779 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" Sep 11 16:29:16.862219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.861775 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd" event={"ID":"b1e63add-64d3-426b-86a5-f0c99bf23920","Type":"ContainerDied","Data":"19bedf72c568694b24e470fcb1ad8bf5abb9d527a812a00cfb46c33212a18eee"} Sep 11 16:29:16.862219 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.861905 2553 scope.go:117] "RemoveContainer" containerID="19d5bf5497f219a41635c07ae03f85fcf541b5e683d4e81f0ed4b37c103dca15" Sep 11 16:29:16.882933 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.882910 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd"] Sep 11 16:29:16.886497 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:16.886477 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8975a-67db9b666-tz2kd"] Sep 11 16:29:18.076353 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:18.076325 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" path="/var/lib/kubelet/pods/b1e63add-64d3-426b-86a5-f0c99bf23920/volumes" Sep 11 16:29:18.853037 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:18.853010 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:21.602140 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.602111 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx"] Sep 11 16:29:21.602503 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.602340 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" containerID="cri-o://f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0" gracePeriod=30 Sep 11 16:29:21.621986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.621960 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:21.751888 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.751856 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:21.838536 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.838507 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:21.907878 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.907807 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:21.970163 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:21.970135 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:22.070132 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:22.070101 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:23.852125 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:23.852081 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:28.851676 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:28.851642 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:33.852116 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:33.852078 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:33.852576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:33.852191 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:38.852382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:38.852337 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:43.852081 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:43.852036 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:45.955625 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.955590 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv"] Sep 11 16:29:45.956016 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.955899 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" Sep 11 16:29:45.956016 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.955911 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" Sep 11 16:29:45.956016 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.955963 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1e63add-64d3-426b-86a5-f0c99bf23920" containerName="switch-graph-8975a" Sep 11 16:29:45.959775 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.959754 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:45.962215 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.962195 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-9c92a-serving-cert\"" Sep 11 16:29:45.962336 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.962290 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-9c92a-kube-rbac-proxy-sar-config\"" Sep 11 16:29:45.967888 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.967866 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv"] Sep 11 16:29:45.969933 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:45.969904 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:46.070036 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.069999 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.070206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.070054 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/473a3212-1952-40cc-8afa-dafe84b489f1-openshift-service-ca-bundle\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.170871 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.170832 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.170871 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.170879 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/473a3212-1952-40cc-8afa-dafe84b489f1-openshift-service-ca-bundle\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.171088 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:29:46.170982 2553 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-9c92a-serving-cert: secret "switch-graph-9c92a-serving-cert" not found Sep 11 16:29:46.171088 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:29:46.171051 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls podName:473a3212-1952-40cc-8afa-dafe84b489f1 nodeName:}" failed. No retries permitted until 2026-09-11 16:29:46.671030689 +0000 UTC m=+743.086128211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls") pod "switch-graph-9c92a-7458554479-n8fnv" (UID: "473a3212-1952-40cc-8afa-dafe84b489f1") : secret "switch-graph-9c92a-serving-cert" not found Sep 11 16:29:46.171462 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.171445 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/473a3212-1952-40cc-8afa-dafe84b489f1-openshift-service-ca-bundle\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.245382 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.245307 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:29:46.675449 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.675356 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.677896 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.677878 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls\") pod \"switch-graph-9c92a-7458554479-n8fnv\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:46.870532 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:46.870503 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:47.140194 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:47.140171 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv"] Sep 11 16:29:47.142819 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:29:47.142789 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473a3212_1952_40cc_8afa_dafe84b489f1.slice/crio-92f2880714ca781d57a81f4a4b3de20e9c080e2c28a90f0c93a3b54bfa45b3f3 WatchSource:0}: Error finding container 92f2880714ca781d57a81f4a4b3de20e9c080e2c28a90f0c93a3b54bfa45b3f3: Status 404 returned error can't find the container with id 92f2880714ca781d57a81f4a4b3de20e9c080e2c28a90f0c93a3b54bfa45b3f3 Sep 11 16:29:47.983061 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:47.983027 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" event={"ID":"473a3212-1952-40cc-8afa-dafe84b489f1","Type":"ContainerStarted","Data":"c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e"} Sep 11 16:29:47.983249 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:47.983076 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" event={"ID":"473a3212-1952-40cc-8afa-dafe84b489f1","Type":"ContainerStarted","Data":"92f2880714ca781d57a81f4a4b3de20e9c080e2c28a90f0c93a3b54bfa45b3f3"} Sep 11 16:29:47.983249 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:47.983117 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:29:47.998980 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:47.998931 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podStartSLOduration=2.998912241 podStartE2EDuration="2.998912241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:47.99794346 +0000 UTC m=+744.413041003" watchObservedRunningTime="2026-09-11 16:29:47.998912241 +0000 UTC m=+744.414009784" Sep 11 16:29:48.852494 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:48.852455 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:29:51.746753 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.746728 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:51.818583 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.818558 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls\") pod \"80d98814-f0dd-447f-9957-0410f03b41c0\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " Sep 11 16:29:51.818757 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.818715 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80d98814-f0dd-447f-9957-0410f03b41c0-openshift-service-ca-bundle\") pod \"80d98814-f0dd-447f-9957-0410f03b41c0\" (UID: \"80d98814-f0dd-447f-9957-0410f03b41c0\") " Sep 11 16:29:51.819044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.819018 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d98814-f0dd-447f-9957-0410f03b41c0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "80d98814-f0dd-447f-9957-0410f03b41c0" (UID: "80d98814-f0dd-447f-9957-0410f03b41c0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:29:51.820563 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.820545 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "80d98814-f0dd-447f-9957-0410f03b41c0" (UID: "80d98814-f0dd-447f-9957-0410f03b41c0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:29:51.919528 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.919456 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80d98814-f0dd-447f-9957-0410f03b41c0-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:29:51.919528 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.919479 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d98814-f0dd-447f-9957-0410f03b41c0-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:29:51.999046 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.999014 2553 generic.go:356] "Generic (PLEG): container finished" podID="80d98814-f0dd-447f-9957-0410f03b41c0" containerID="f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0" exitCode=0 Sep 11 16:29:51.999196 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.999069 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" Sep 11 16:29:51.999196 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.999079 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" event={"ID":"80d98814-f0dd-447f-9957-0410f03b41c0","Type":"ContainerDied","Data":"f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0"} Sep 11 16:29:51.999196 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.999120 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx" event={"ID":"80d98814-f0dd-447f-9957-0410f03b41c0","Type":"ContainerDied","Data":"f62cd72da6e2489cd753a1f07ba015375396294ad181e2888c6c374ddfc13124"} Sep 11 16:29:51.999196 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:51.999144 2553 scope.go:117] "RemoveContainer" containerID="f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0" Sep 11 16:29:52.007576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:52.007560 2553 scope.go:117] "RemoveContainer" containerID="f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0" Sep 11 16:29:52.007883 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:29:52.007856 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0\": container with ID starting with f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0 not found: ID does not exist" containerID="f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0" Sep 11 16:29:52.007945 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:52.007892 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0"} err="failed to get container status \"f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0\": rpc error: code = NotFound desc = could not find container \"f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0\": container with ID starting with f54f1b57bce30fc38cd1036a2e1742dcd31bbd54bc6d34fc752df8328c71d2d0 not found: ID does not exist" Sep 11 16:29:52.019218 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:52.019195 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx"] Sep 11 16:29:52.022908 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:52.022890 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-6f97b8c58f-jd8dx"] Sep 11 16:29:52.076501 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:52.076473 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" path="/var/lib/kubelet/pods/80d98814-f0dd-447f-9957-0410f03b41c0/volumes" Sep 11 16:29:53.991851 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:29:53.991822 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:30:21.803772 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.803737 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57"] Sep 11 16:30:21.804406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.804085 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" Sep 11 16:30:21.804406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.804100 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" Sep 11 16:30:21.804406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.804150 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d98814-f0dd-447f-9957-0410f03b41c0" containerName="model-chainer" Sep 11 16:30:21.807088 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.807070 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:21.809450 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.809420 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-ff919-serving-cert\"" Sep 11 16:30:21.809585 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.809490 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-ff919-kube-rbac-proxy-sar-config\"" Sep 11 16:30:21.809744 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.809728 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:30:21.815563 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.815540 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57"] Sep 11 16:30:21.969301 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.969256 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63cf65da-9c6e-49cd-88b6-2f8b973ec491-proxy-tls\") pod \"sequence-graph-ff919-c4589574f-j5k57\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:21.969480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:21.969309 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63cf65da-9c6e-49cd-88b6-2f8b973ec491-openshift-service-ca-bundle\") pod \"sequence-graph-ff919-c4589574f-j5k57\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:22.040012 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.039981 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:30:22.070119 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.070056 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63cf65da-9c6e-49cd-88b6-2f8b973ec491-proxy-tls\") pod \"sequence-graph-ff919-c4589574f-j5k57\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:22.070119 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.070093 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63cf65da-9c6e-49cd-88b6-2f8b973ec491-openshift-service-ca-bundle\") pod \"sequence-graph-ff919-c4589574f-j5k57\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:22.070733 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.070716 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63cf65da-9c6e-49cd-88b6-2f8b973ec491-openshift-service-ca-bundle\") pod \"sequence-graph-ff919-c4589574f-j5k57\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:22.072577 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.072557 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63cf65da-9c6e-49cd-88b6-2f8b973ec491-proxy-tls\") pod \"sequence-graph-ff919-c4589574f-j5k57\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:22.117957 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.117928 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:22.386529 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:22.386458 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57"] Sep 11 16:30:22.389530 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:30:22.389506 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cf65da_9c6e_49cd_88b6_2f8b973ec491.slice/crio-eecb6fabcc53f5d3093f029b8418493af8e81e5f7597b59ddc1acefed2ab6204 WatchSource:0}: Error finding container eecb6fabcc53f5d3093f029b8418493af8e81e5f7597b59ddc1acefed2ab6204: Status 404 returned error can't find the container with id eecb6fabcc53f5d3093f029b8418493af8e81e5f7597b59ddc1acefed2ab6204 Sep 11 16:30:23.125787 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:23.125752 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" event={"ID":"63cf65da-9c6e-49cd-88b6-2f8b973ec491","Type":"ContainerStarted","Data":"3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50"} Sep 11 16:30:23.126282 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:23.125803 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" event={"ID":"63cf65da-9c6e-49cd-88b6-2f8b973ec491","Type":"ContainerStarted","Data":"eecb6fabcc53f5d3093f029b8418493af8e81e5f7597b59ddc1acefed2ab6204"} Sep 11 16:30:23.126282 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:23.125871 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:30:23.141628 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:23.141580 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podStartSLOduration=2.141566105 podStartE2EDuration="2.141566105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:30:23.140503692 +0000 UTC m=+779.555601236" watchObservedRunningTime="2026-09-11 16:30:23.141566105 +0000 UTC m=+779.556663659" Sep 11 16:30:29.134500 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:30:29.134471 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:32:46.642858 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:32:46.642828 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:32:46.731230 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:32:46.731197 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:34:59.480001 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:34:59.479964 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:36:24.808834 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:36:24.808800 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:37:44.858148 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:37:44.858117 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:00.642507 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:00.642473 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv"] Sep 11 16:38:00.643055 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:00.642757 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" containerID="cri-o://c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e" gracePeriod=30 Sep 11 16:38:00.673132 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:00.673097 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:00.798830 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:00.798797 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:00.886075 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:00.886041 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:01.048259 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:01.048223 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:01.136402 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:01.136373 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:01.362428 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:01.362355 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:03.991182 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:03.991143 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:08.990900 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:08.990852 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:13.990895 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:13.990848 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:13.991276 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:13.990985 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:38:18.991052 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:18.990999 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:23.990206 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:23.990162 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:28.990509 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:28.990469 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:30.796556 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.796526 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:38:30.847904 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.847867 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls\") pod \"473a3212-1952-40cc-8afa-dafe84b489f1\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " Sep 11 16:38:30.848092 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.847992 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/473a3212-1952-40cc-8afa-dafe84b489f1-openshift-service-ca-bundle\") pod \"473a3212-1952-40cc-8afa-dafe84b489f1\" (UID: \"473a3212-1952-40cc-8afa-dafe84b489f1\") " Sep 11 16:38:30.848363 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.848335 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/473a3212-1952-40cc-8afa-dafe84b489f1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "473a3212-1952-40cc-8afa-dafe84b489f1" (UID: "473a3212-1952-40cc-8afa-dafe84b489f1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:38:30.850084 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.850063 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "473a3212-1952-40cc-8afa-dafe84b489f1" (UID: "473a3212-1952-40cc-8afa-dafe84b489f1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:38:30.948638 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.948604 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/473a3212-1952-40cc-8afa-dafe84b489f1-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:38:30.948806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.948643 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/473a3212-1952-40cc-8afa-dafe84b489f1-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:38:30.952775 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.952739 2553 generic.go:356] "Generic (PLEG): container finished" podID="473a3212-1952-40cc-8afa-dafe84b489f1" containerID="c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e" exitCode=0 Sep 11 16:38:30.952775 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.952776 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" event={"ID":"473a3212-1952-40cc-8afa-dafe84b489f1","Type":"ContainerDied","Data":"c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e"} Sep 11 16:38:30.952944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.952787 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" Sep 11 16:38:30.952944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.952843 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv" event={"ID":"473a3212-1952-40cc-8afa-dafe84b489f1","Type":"ContainerDied","Data":"92f2880714ca781d57a81f4a4b3de20e9c080e2c28a90f0c93a3b54bfa45b3f3"} Sep 11 16:38:30.952944 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.952865 2553 scope.go:117] "RemoveContainer" containerID="c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e" Sep 11 16:38:30.961766 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.961743 2553 scope.go:117] "RemoveContainer" containerID="c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e" Sep 11 16:38:30.962057 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:38:30.962040 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e\": container with ID starting with c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e not found: ID does not exist" containerID="c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e" Sep 11 16:38:30.962098 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.962066 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e"} err="failed to get container status \"c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e\": rpc error: code = NotFound desc = could not find container \"c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e\": container with ID starting with c6a2dcc550a9a8dc20ea77e0738ad78395ef1b71a183fb435b79f13d7e73dd2e not found: ID does not exist" Sep 11 16:38:30.974849 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.974820 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv"] Sep 11 16:38:30.976901 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:30.976877 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9c92a-7458554479-n8fnv"] Sep 11 16:38:32.076459 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:32.076424 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" path="/var/lib/kubelet/pods/473a3212-1952-40cc-8afa-dafe84b489f1/volumes" Sep 11 16:38:36.506990 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.506943 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57"] Sep 11 16:38:36.507368 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.507171 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" containerID="cri-o://3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50" gracePeriod=30 Sep 11 16:38:36.535383 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.535345 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:36.649963 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.649924 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:36.720789 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.720759 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:36.795710 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.795607 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:36.870812 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:36.870783 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:37.183501 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:37.183408 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:37.282357 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:37.282330 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:38:39.133927 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:39.133891 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:44.133376 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:44.133330 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:49.133520 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:49.133473 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:49.133943 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:49.133581 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:38:54.133576 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:54.133537 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:38:59.133020 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:38:59.132980 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:00.874092 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.874053 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d"] Sep 11 16:39:00.874469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.874362 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" Sep 11 16:39:00.874469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.874374 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" Sep 11 16:39:00.874469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.874420 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="473a3212-1952-40cc-8afa-dafe84b489f1" containerName="switch-graph-9c92a" Sep 11 16:39:00.877240 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.877220 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:00.879882 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.879862 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-a9581-kube-rbac-proxy-sar-config\"" Sep 11 16:39:00.879999 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.879913 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-a9581-serving-cert\"" Sep 11 16:39:00.887349 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.887326 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d"] Sep 11 16:39:00.889517 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.889493 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:00.992991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.992951 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:00.994152 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.994133 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-openshift-service-ca-bundle\") pod \"ensemble-graph-a9581-694df647d-nn95d\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:00.994222 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:00.994162 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-proxy-tls\") pod \"ensemble-graph-a9581-694df647d-nn95d\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:01.095145 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:01.095107 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-openshift-service-ca-bundle\") pod \"ensemble-graph-a9581-694df647d-nn95d\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:01.095145 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:01.095147 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-proxy-tls\") pod \"ensemble-graph-a9581-694df647d-nn95d\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:01.095775 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:01.095753 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-openshift-service-ca-bundle\") pod \"ensemble-graph-a9581-694df647d-nn95d\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:01.097768 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:01.097748 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-proxy-tls\") pod \"ensemble-graph-a9581-694df647d-nn95d\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:01.187750 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:01.187652 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:01.456843 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:01.456809 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d"] Sep 11 16:39:01.459455 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:39:01.459428 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e34d8cf_5311_4868_9966_f0a1ff81e0e0.slice/crio-dc550f33b2ffd56aae701ca46cca296ff9030e8c9672f045b1ef7d99bffe5eb7 WatchSource:0}: Error finding container dc550f33b2ffd56aae701ca46cca296ff9030e8c9672f045b1ef7d99bffe5eb7: Status 404 returned error can't find the container with id dc550f33b2ffd56aae701ca46cca296ff9030e8c9672f045b1ef7d99bffe5eb7 Sep 11 16:39:02.067515 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:02.067476 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" event={"ID":"5e34d8cf-5311-4868-9966-f0a1ff81e0e0","Type":"ContainerStarted","Data":"0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564"} Sep 11 16:39:02.067904 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:02.067538 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" event={"ID":"5e34d8cf-5311-4868-9966-f0a1ff81e0e0","Type":"ContainerStarted","Data":"dc550f33b2ffd56aae701ca46cca296ff9030e8c9672f045b1ef7d99bffe5eb7"} Sep 11 16:39:02.067904 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:02.067613 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:02.084202 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:02.084143 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podStartSLOduration=2.084123004 podStartE2EDuration="2.084123004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:39:02.083438522 +0000 UTC m=+1298.498536068" watchObservedRunningTime="2026-09-11 16:39:02.084123004 +0000 UTC m=+1298.499220552" Sep 11 16:39:04.133075 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:04.133029 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:06.653301 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.653273 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:39:06.736610 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.736575 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63cf65da-9c6e-49cd-88b6-2f8b973ec491-proxy-tls\") pod \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " Sep 11 16:39:06.736802 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.736619 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63cf65da-9c6e-49cd-88b6-2f8b973ec491-openshift-service-ca-bundle\") pod \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\" (UID: \"63cf65da-9c6e-49cd-88b6-2f8b973ec491\") " Sep 11 16:39:06.737020 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.736998 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63cf65da-9c6e-49cd-88b6-2f8b973ec491-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "63cf65da-9c6e-49cd-88b6-2f8b973ec491" (UID: "63cf65da-9c6e-49cd-88b6-2f8b973ec491"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:39:06.738826 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.738802 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63cf65da-9c6e-49cd-88b6-2f8b973ec491-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "63cf65da-9c6e-49cd-88b6-2f8b973ec491" (UID: "63cf65da-9c6e-49cd-88b6-2f8b973ec491"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:39:06.838051 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.837974 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/63cf65da-9c6e-49cd-88b6-2f8b973ec491-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:39:06.838051 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:06.838003 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63cf65da-9c6e-49cd-88b6-2f8b973ec491-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:39:07.086986 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.086951 2553 generic.go:356] "Generic (PLEG): container finished" podID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerID="3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50" exitCode=0 Sep 11 16:39:07.087134 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.087001 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" Sep 11 16:39:07.087134 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.087030 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" event={"ID":"63cf65da-9c6e-49cd-88b6-2f8b973ec491","Type":"ContainerDied","Data":"3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50"} Sep 11 16:39:07.087134 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.087074 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57" event={"ID":"63cf65da-9c6e-49cd-88b6-2f8b973ec491","Type":"ContainerDied","Data":"eecb6fabcc53f5d3093f029b8418493af8e81e5f7597b59ddc1acefed2ab6204"} Sep 11 16:39:07.087134 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.087094 2553 scope.go:117] "RemoveContainer" containerID="3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50" Sep 11 16:39:07.095851 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.095831 2553 scope.go:117] "RemoveContainer" containerID="3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50" Sep 11 16:39:07.096149 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:39:07.096128 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50\": container with ID starting with 3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50 not found: ID does not exist" containerID="3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50" Sep 11 16:39:07.096226 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.096163 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50"} err="failed to get container status \"3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50\": rpc error: code = NotFound desc = could not find container \"3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50\": container with ID starting with 3c384f591ea041e723177fefdca35d4d1ca71ef38958e4b9e95d1178118f9e50 not found: ID does not exist" Sep 11 16:39:07.108544 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.108521 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57"] Sep 11 16:39:07.112096 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:07.112064 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ff919-c4589574f-j5k57"] Sep 11 16:39:08.076312 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:08.076282 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" path="/var/lib/kubelet/pods/63cf65da-9c6e-49cd-88b6-2f8b973ec491/volumes" Sep 11 16:39:08.078945 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:08.078918 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:10.951954 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:10.951924 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d"] Sep 11 16:39:10.952372 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:10.952106 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" containerID="cri-o://0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564" gracePeriod=30 Sep 11 16:39:10.978780 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:10.978742 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:11.078721 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:11.078665 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:11.146439 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:11.146369 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:11.232192 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:11.232071 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:11.462462 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:11.462431 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:11.676952 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:11.676874 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:13.075343 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:13.075294 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:18.075807 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:18.075761 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:23.075437 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:23.075395 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:23.075928 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:23.075523 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:28.076292 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:28.076249 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:33.075615 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:33.075575 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:36.703798 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.703761 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8"] Sep 11 16:39:36.704297 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.704098 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" Sep 11 16:39:36.704297 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.704112 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" Sep 11 16:39:36.704297 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.704185 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="63cf65da-9c6e-49cd-88b6-2f8b973ec491" containerName="sequence-graph-ff919" Sep 11 16:39:36.707315 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.707286 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:36.708375 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.708359 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:36.710784 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.710759 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-e5408-serving-cert\"" Sep 11 16:39:36.710898 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.710800 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-e5408-kube-rbac-proxy-sar-config\"" Sep 11 16:39:36.718592 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.718573 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8"] Sep 11 16:39:36.774991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.774949 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-openshift-service-ca-bundle\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:36.775151 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.775024 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:36.875739 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.875675 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-openshift-service-ca-bundle\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:36.875904 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.875778 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:36.875904 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:39:36.875886 2553 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-e5408-serving-cert: secret "sequence-graph-e5408-serving-cert" not found Sep 11 16:39:36.875978 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:39:36.875947 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls podName:6c2e8e73-fc91-46ca-b629-ea76fec6d58a nodeName:}" failed. No retries permitted until 2026-09-11 16:39:37.375930234 +0000 UTC m=+1333.791027755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls") pod "sequence-graph-e5408-66fb7f646b-d99b8" (UID: "6c2e8e73-fc91-46ca-b629-ea76fec6d58a") : secret "sequence-graph-e5408-serving-cert" not found Sep 11 16:39:36.876306 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:36.876287 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-openshift-service-ca-bundle\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:37.075625 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:37.075540 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:37.380083 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:37.379989 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:37.382569 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:37.382545 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls\") pod \"sequence-graph-e5408-66fb7f646b-d99b8\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:37.619652 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:37.619615 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:37.887869 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:37.887818 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8"] Sep 11 16:39:37.893584 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:39:37.893557 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2e8e73_fc91_46ca_b629_ea76fec6d58a.slice/crio-25d9e02a9180bbc9a5640b107564de4e5e08f21962a6aa8265133ac416cf199c WatchSource:0}: Error finding container 25d9e02a9180bbc9a5640b107564de4e5e08f21962a6aa8265133ac416cf199c: Status 404 returned error can't find the container with id 25d9e02a9180bbc9a5640b107564de4e5e08f21962a6aa8265133ac416cf199c Sep 11 16:39:38.076049 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:38.076014 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:38.204790 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:38.204757 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" event={"ID":"6c2e8e73-fc91-46ca-b629-ea76fec6d58a","Type":"ContainerStarted","Data":"2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e"} Sep 11 16:39:38.204962 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:38.204810 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" event={"ID":"6c2e8e73-fc91-46ca-b629-ea76fec6d58a","Type":"ContainerStarted","Data":"25d9e02a9180bbc9a5640b107564de4e5e08f21962a6aa8265133ac416cf199c"} Sep 11 16:39:38.204962 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:38.204862 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:38.221528 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:38.221475 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podStartSLOduration=2.22146047 podStartE2EDuration="2.22146047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:39:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:39:38.220897382 +0000 UTC m=+1334.635994926" watchObservedRunningTime="2026-09-11 16:39:38.22146047 +0000 UTC m=+1334.636558012" Sep 11 16:39:41.093703 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.093657 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:41.212991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.212886 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-openshift-service-ca-bundle\") pod \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " Sep 11 16:39:41.212991 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.212967 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-proxy-tls\") pod \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\" (UID: \"5e34d8cf-5311-4868-9966-f0a1ff81e0e0\") " Sep 11 16:39:41.213326 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.213301 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5e34d8cf-5311-4868-9966-f0a1ff81e0e0" (UID: "5e34d8cf-5311-4868-9966-f0a1ff81e0e0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:39:41.215224 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.215196 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5e34d8cf-5311-4868-9966-f0a1ff81e0e0" (UID: "5e34d8cf-5311-4868-9966-f0a1ff81e0e0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:39:41.217934 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.217897 2553 generic.go:356] "Generic (PLEG): container finished" podID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerID="0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564" exitCode=0 Sep 11 16:39:41.218048 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.217942 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" event={"ID":"5e34d8cf-5311-4868-9966-f0a1ff81e0e0","Type":"ContainerDied","Data":"0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564"} Sep 11 16:39:41.218048 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.217962 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" Sep 11 16:39:41.218048 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.218023 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d" event={"ID":"5e34d8cf-5311-4868-9966-f0a1ff81e0e0","Type":"ContainerDied","Data":"dc550f33b2ffd56aae701ca46cca296ff9030e8c9672f045b1ef7d99bffe5eb7"} Sep 11 16:39:41.218181 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.218054 2553 scope.go:117] "RemoveContainer" containerID="0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564" Sep 11 16:39:41.228198 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.227974 2553 scope.go:117] "RemoveContainer" containerID="0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564" Sep 11 16:39:41.228380 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:39:41.228353 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564\": container with ID starting with 0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564 not found: ID does not exist" containerID="0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564" Sep 11 16:39:41.228459 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.228393 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564"} err="failed to get container status \"0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564\": rpc error: code = NotFound desc = could not find container \"0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564\": container with ID starting with 0f89c81587cc0d964ccc27f8c28624086fff52a7300ad794f25e71be9e5f3564 not found: ID does not exist" Sep 11 16:39:41.240849 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.240811 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d"] Sep 11 16:39:41.245039 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.245014 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-a9581-694df647d-nn95d"] Sep 11 16:39:41.314480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.314443 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:39:41.314480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:41.314472 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5e34d8cf-5311-4868-9966-f0a1ff81e0e0-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:39:42.076304 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:42.076264 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" path="/var/lib/kubelet/pods/5e34d8cf-5311-4868-9966-f0a1ff81e0e0/volumes" Sep 11 16:39:44.214064 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:44.214031 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:39:46.775784 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:46.775707 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8"] Sep 11 16:39:46.776189 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:46.775877 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" containerID="cri-o://2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e" gracePeriod=30 Sep 11 16:39:46.796776 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:46.796747 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:46.884675 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:46.884640 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:46.959866 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:46.959835 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:47.056898 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:47.056816 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:47.133962 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:47.133932 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:47.212458 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:47.212432 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:47.312310 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:47.312221 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:39:49.212653 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:49.212615 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:54.212148 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:54.212110 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:59.212446 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:59.212405 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:39:59.212931 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:39:59.212502 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:40:04.212957 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:04.212920 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:40:09.213349 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:09.213310 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:40:11.172571 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.172528 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9"] Sep 11 16:40:11.173217 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.173197 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" Sep 11 16:40:11.173288 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.173220 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" Sep 11 16:40:11.173400 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.173386 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e34d8cf-5311-4868-9966-f0a1ff81e0e0" containerName="ensemble-graph-a9581" Sep 11 16:40:11.177060 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.177032 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.179751 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.179730 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1ab9e-kube-rbac-proxy-sar-config\"" Sep 11 16:40:11.179848 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.179730 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1ab9e-serving-cert\"" Sep 11 16:40:11.182985 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.182961 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9"] Sep 11 16:40:11.191806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.191774 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:40:11.253414 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.253378 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d63cf-7639-433c-9ee1-9d85cdd2c318-openshift-service-ca-bundle\") pod \"ensemble-graph-1ab9e-567bbdc5cc-77rv9\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.253414 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.253418 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/705d63cf-7639-433c-9ee1-9d85cdd2c318-proxy-tls\") pod \"ensemble-graph-1ab9e-567bbdc5cc-77rv9\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.354719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.354658 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d63cf-7639-433c-9ee1-9d85cdd2c318-openshift-service-ca-bundle\") pod \"ensemble-graph-1ab9e-567bbdc5cc-77rv9\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.354719 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.354719 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/705d63cf-7639-433c-9ee1-9d85cdd2c318-proxy-tls\") pod \"ensemble-graph-1ab9e-567bbdc5cc-77rv9\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.355444 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.355396 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d63cf-7639-433c-9ee1-9d85cdd2c318-openshift-service-ca-bundle\") pod \"ensemble-graph-1ab9e-567bbdc5cc-77rv9\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.357248 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.357227 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/705d63cf-7639-433c-9ee1-9d85cdd2c318-proxy-tls\") pod \"ensemble-graph-1ab9e-567bbdc5cc-77rv9\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.488471 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.488441 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:11.765783 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:11.765707 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9"] Sep 11 16:40:11.769339 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:40:11.769313 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod705d63cf_7639_433c_9ee1_9d85cdd2c318.slice/crio-ce5b199c7a01932f35f297ce950b4507b81fc73d2e190a3f7a2e9994b28c7eea WatchSource:0}: Error finding container ce5b199c7a01932f35f297ce950b4507b81fc73d2e190a3f7a2e9994b28c7eea: Status 404 returned error can't find the container with id ce5b199c7a01932f35f297ce950b4507b81fc73d2e190a3f7a2e9994b28c7eea Sep 11 16:40:12.279506 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:12.279472 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:40:12.330997 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:12.330968 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" event={"ID":"705d63cf-7639-433c-9ee1-9d85cdd2c318","Type":"ContainerStarted","Data":"0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb"} Sep 11 16:40:12.331164 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:12.331015 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" event={"ID":"705d63cf-7639-433c-9ee1-9d85cdd2c318","Type":"ContainerStarted","Data":"ce5b199c7a01932f35f297ce950b4507b81fc73d2e190a3f7a2e9994b28c7eea"} Sep 11 16:40:12.331164 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:12.331054 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:12.348981 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:12.348932 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podStartSLOduration=1.348916952 podStartE2EDuration="1.348916952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:40:12.348292894 +0000 UTC m=+1368.763390450" watchObservedRunningTime="2026-09-11 16:40:12.348916952 +0000 UTC m=+1368.764014496" Sep 11 16:40:14.212311 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:14.212265 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:40:16.919100 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:16.919076 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:40:16.994532 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:16.994500 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-openshift-service-ca-bundle\") pod \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " Sep 11 16:40:16.994532 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:16.994535 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls\") pod \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\" (UID: \"6c2e8e73-fc91-46ca-b629-ea76fec6d58a\") " Sep 11 16:40:16.994926 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:16.994909 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6c2e8e73-fc91-46ca-b629-ea76fec6d58a" (UID: "6c2e8e73-fc91-46ca-b629-ea76fec6d58a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:40:16.996699 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:16.996666 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6c2e8e73-fc91-46ca-b629-ea76fec6d58a" (UID: "6c2e8e73-fc91-46ca-b629-ea76fec6d58a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:40:17.095460 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.095375 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:40:17.095460 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.095408 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6c2e8e73-fc91-46ca-b629-ea76fec6d58a-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:40:17.351469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.351372 2553 generic.go:356] "Generic (PLEG): container finished" podID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerID="2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e" exitCode=0 Sep 11 16:40:17.351469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.351422 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" Sep 11 16:40:17.351469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.351446 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" event={"ID":"6c2e8e73-fc91-46ca-b629-ea76fec6d58a","Type":"ContainerDied","Data":"2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e"} Sep 11 16:40:17.351785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.351491 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8" event={"ID":"6c2e8e73-fc91-46ca-b629-ea76fec6d58a","Type":"ContainerDied","Data":"25d9e02a9180bbc9a5640b107564de4e5e08f21962a6aa8265133ac416cf199c"} Sep 11 16:40:17.351785 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.351514 2553 scope.go:117] "RemoveContainer" containerID="2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e" Sep 11 16:40:17.361147 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.361127 2553 scope.go:117] "RemoveContainer" containerID="2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e" Sep 11 16:40:17.361494 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:40:17.361471 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e\": container with ID starting with 2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e not found: ID does not exist" containerID="2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e" Sep 11 16:40:17.361601 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.361506 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e"} err="failed to get container status \"2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e\": rpc error: code = NotFound desc = could not find container \"2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e\": container with ID starting with 2814e45af6464217ef04a6a90d99127dfbdfd9d084e00cbf6fe272b0cc2da20e not found: ID does not exist" Sep 11 16:40:17.379572 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.379544 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8"] Sep 11 16:40:17.385976 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:17.385940 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e5408-66fb7f646b-d99b8"] Sep 11 16:40:18.076383 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:18.076352 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" path="/var/lib/kubelet/pods/6c2e8e73-fc91-46ca-b629-ea76fec6d58a/volumes" Sep 11 16:40:18.341268 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:18.341183 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:40:46.999898 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:46.999870 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm"] Sep 11 16:40:47.000413 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.000191 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" Sep 11 16:40:47.000413 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.000204 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" Sep 11 16:40:47.000413 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.000276 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c2e8e73-fc91-46ca-b629-ea76fec6d58a" containerName="sequence-graph-e5408" Sep 11 16:40:47.000653 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.000623 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:40:47.004758 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.004739 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.007585 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.007566 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-ea3c5-serving-cert\"" Sep 11 16:40:47.007710 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.007569 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-ea3c5-kube-rbac-proxy-sar-config\"" Sep 11 16:40:47.014841 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.014820 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm"] Sep 11 16:40:47.136979 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.136942 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-openshift-service-ca-bundle\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.137150 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.137002 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.237482 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.237436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.237664 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.237534 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-openshift-service-ca-bundle\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.237664 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:40:47.237591 2553 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-ea3c5-serving-cert: secret "sequence-graph-ea3c5-serving-cert" not found Sep 11 16:40:47.237765 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:40:47.237667 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls podName:bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a nodeName:}" failed. No retries permitted until 2026-09-11 16:40:47.737649451 +0000 UTC m=+1404.152746973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls") pod "sequence-graph-ea3c5-5d669fffd4-pqsgm" (UID: "bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a") : secret "sequence-graph-ea3c5-serving-cert" not found Sep 11 16:40:47.238305 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.238283 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-openshift-service-ca-bundle\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.303480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.303341 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:40:47.741605 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.741571 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.744131 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.744104 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls\") pod \"sequence-graph-ea3c5-5d669fffd4-pqsgm\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:47.914200 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:47.914166 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:48.182139 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:48.182117 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm"] Sep 11 16:40:48.184519 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:40:48.184492 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbaf5cf1_a97b_40f5_b77c_0c8365aa5e3a.slice/crio-f7de1afd466210ccc748fdf7fd8df0ea3f6f991fcf5b1306315373d64d03d244 WatchSource:0}: Error finding container f7de1afd466210ccc748fdf7fd8df0ea3f6f991fcf5b1306315373d64d03d244: Status 404 returned error can't find the container with id f7de1afd466210ccc748fdf7fd8df0ea3f6f991fcf5b1306315373d64d03d244 Sep 11 16:40:48.464911 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:48.464872 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" event={"ID":"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a","Type":"ContainerStarted","Data":"a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6"} Sep 11 16:40:48.465087 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:48.464934 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" event={"ID":"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a","Type":"ContainerStarted","Data":"f7de1afd466210ccc748fdf7fd8df0ea3f6f991fcf5b1306315373d64d03d244"} Sep 11 16:40:48.465087 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:48.465050 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:48.482053 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:48.481982 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podStartSLOduration=2.481960715 podStartE2EDuration="2.481960715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:40:48.480769329 +0000 UTC m=+1404.895866876" watchObservedRunningTime="2026-09-11 16:40:48.481960715 +0000 UTC m=+1404.897058259" Sep 11 16:40:54.474547 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:54.474522 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:40:55.673729 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:40:55.673674 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:42:53.575201 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:42:53.575108 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:43:10.957371 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:43:10.957338 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:43:14.222408 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:43:14.222363 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:43:25.893158 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:43:25.893115 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:48:25.877182 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:25.877145 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9"] Sep 11 16:48:25.877783 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:25.877404 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" containerID="cri-o://0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb" gracePeriod=30 Sep 11 16:48:25.930262 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:25.930159 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:48:27.018469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:27.018425 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:48:28.338769 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:28.338725 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:48:33.338837 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:33.338795 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:48:38.339544 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:38.339505 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:48:38.340009 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:38.339623 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:48:43.339276 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:43.339237 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:48:48.339432 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:48.339340 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:48:53.339592 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:53.339559 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:48:56.022997 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.022967 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:48:56.209525 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.209482 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/705d63cf-7639-433c-9ee1-9d85cdd2c318-proxy-tls\") pod \"705d63cf-7639-433c-9ee1-9d85cdd2c318\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " Sep 11 16:48:56.209525 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.209528 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d63cf-7639-433c-9ee1-9d85cdd2c318-openshift-service-ca-bundle\") pod \"705d63cf-7639-433c-9ee1-9d85cdd2c318\" (UID: \"705d63cf-7639-433c-9ee1-9d85cdd2c318\") " Sep 11 16:48:56.209943 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.209909 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/705d63cf-7639-433c-9ee1-9d85cdd2c318-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "705d63cf-7639-433c-9ee1-9d85cdd2c318" (UID: "705d63cf-7639-433c-9ee1-9d85cdd2c318"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:48:56.211716 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.211694 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/705d63cf-7639-433c-9ee1-9d85cdd2c318-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "705d63cf-7639-433c-9ee1-9d85cdd2c318" (UID: "705d63cf-7639-433c-9ee1-9d85cdd2c318"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:48:56.273764 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.273735 2553 generic.go:356] "Generic (PLEG): container finished" podID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerID="0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb" exitCode=0 Sep 11 16:48:56.273935 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.273775 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" Sep 11 16:48:56.273935 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.273803 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" event={"ID":"705d63cf-7639-433c-9ee1-9d85cdd2c318","Type":"ContainerDied","Data":"0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb"} Sep 11 16:48:56.273935 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.273845 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9" event={"ID":"705d63cf-7639-433c-9ee1-9d85cdd2c318","Type":"ContainerDied","Data":"ce5b199c7a01932f35f297ce950b4507b81fc73d2e190a3f7a2e9994b28c7eea"} Sep 11 16:48:56.273935 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.273866 2553 scope.go:117] "RemoveContainer" containerID="0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb" Sep 11 16:48:56.282245 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.282225 2553 scope.go:117] "RemoveContainer" containerID="0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb" Sep 11 16:48:56.282540 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:48:56.282521 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb\": container with ID starting with 0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb not found: ID does not exist" containerID="0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb" Sep 11 16:48:56.282602 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.282549 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb"} err="failed to get container status \"0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb\": rpc error: code = NotFound desc = could not find container \"0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb\": container with ID starting with 0323da72786d5edd4dc3e9e2d4f0dc72f4e35e89f6c457bb55deba910f724cdb not found: ID does not exist" Sep 11 16:48:56.294137 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.294105 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9"] Sep 11 16:48:56.297989 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.297966 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1ab9e-567bbdc5cc-77rv9"] Sep 11 16:48:56.310267 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.310246 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/705d63cf-7639-433c-9ee1-9d85cdd2c318-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:48:56.310363 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:56.310269 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/705d63cf-7639-433c-9ee1-9d85cdd2c318-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:48:58.075869 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:48:58.075840 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" path="/var/lib/kubelet/pods/705d63cf-7639-433c-9ee1-9d85cdd2c318/volumes" Sep 11 16:49:01.603052 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:01.603021 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm"] Sep 11 16:49:01.603547 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:01.603233 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" containerID="cri-o://a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6" gracePeriod=30 Sep 11 16:49:01.626796 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:01.626765 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:01.736742 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:01.736707 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:01.834077 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:01.834046 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:01.947362 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:01.947321 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:02.034480 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:02.034439 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:02.135322 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:02.135291 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:02.236398 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:02.236312 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:04.473265 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:04.473231 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:09.473227 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:09.473189 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:14.472551 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:14.472514 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:14.472950 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:14.472626 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:49:19.472671 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:19.472632 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:24.472819 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:24.472783 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:26.113659 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.113620 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b"] Sep 11 16:49:26.114139 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.114069 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" Sep 11 16:49:26.114139 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.114087 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" Sep 11 16:49:26.114249 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.114175 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="705d63cf-7639-433c-9ee1-9d85cdd2c318" containerName="ensemble-graph-1ab9e" Sep 11 16:49:26.116970 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.116950 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.119432 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.119406 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-ddc34-kube-rbac-proxy-sar-config\"" Sep 11 16:49:26.119558 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.119451 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-ddc34-serving-cert\"" Sep 11 16:49:26.128096 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.128073 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b"] Sep 11 16:49:26.128341 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.128315 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:26.242842 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.242810 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-proxy-tls\") pod \"splitter-graph-ddc34-98b445d6c-9zb8b\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.243007 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.242853 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-openshift-service-ca-bundle\") pod \"splitter-graph-ddc34-98b445d6c-9zb8b\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.279827 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.279793 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:26.343646 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.343605 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-openshift-service-ca-bundle\") pod \"splitter-graph-ddc34-98b445d6c-9zb8b\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.343854 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.343701 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-proxy-tls\") pod \"splitter-graph-ddc34-98b445d6c-9zb8b\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.344319 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.344290 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-openshift-service-ca-bundle\") pod \"splitter-graph-ddc34-98b445d6c-9zb8b\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.346329 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.346312 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-proxy-tls\") pod \"splitter-graph-ddc34-98b445d6c-9zb8b\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.427036 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.426922 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:26.696096 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:26.696066 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b"] Sep 11 16:49:26.699001 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:49:26.698971 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109e6e3d_44c2_4115_9b7d_4bb6fc9ffb5a.slice/crio-a079964b9187fdb3a28c7eddc7b7cb7d06b000a4025b165799b24d1381cabbf9 WatchSource:0}: Error finding container a079964b9187fdb3a28c7eddc7b7cb7d06b000a4025b165799b24d1381cabbf9: Status 404 returned error can't find the container with id a079964b9187fdb3a28c7eddc7b7cb7d06b000a4025b165799b24d1381cabbf9 Sep 11 16:49:27.388871 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:27.388831 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" event={"ID":"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a","Type":"ContainerStarted","Data":"1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f"} Sep 11 16:49:27.389257 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:27.388901 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" event={"ID":"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a","Type":"ContainerStarted","Data":"a079964b9187fdb3a28c7eddc7b7cb7d06b000a4025b165799b24d1381cabbf9"} Sep 11 16:49:27.389257 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:27.388950 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:27.405376 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:27.405297 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podStartSLOduration=1.405284841 podStartE2EDuration="1.405284841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:27.4041754 +0000 UTC m=+1923.819272935" watchObservedRunningTime="2026-09-11 16:49:27.405284841 +0000 UTC m=+1923.820382385" Sep 11 16:49:29.473406 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:29.473371 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:31.738645 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.738620 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:49:31.889784 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.889702 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-openshift-service-ca-bundle\") pod \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " Sep 11 16:49:31.889784 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.889740 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls\") pod \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\" (UID: \"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a\") " Sep 11 16:49:31.890100 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.890076 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" (UID: "bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:49:31.891883 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.891866 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" (UID: "bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:49:31.991243 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.991206 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:49:31.991243 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:31.991235 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:49:32.409140 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.409104 2553 generic.go:356] "Generic (PLEG): container finished" podID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerID="a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6" exitCode=0 Sep 11 16:49:32.409313 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.409148 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" Sep 11 16:49:32.409313 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.409189 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" event={"ID":"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a","Type":"ContainerDied","Data":"a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6"} Sep 11 16:49:32.409313 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.409237 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm" event={"ID":"bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a","Type":"ContainerDied","Data":"f7de1afd466210ccc748fdf7fd8df0ea3f6f991fcf5b1306315373d64d03d244"} Sep 11 16:49:32.409313 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.409259 2553 scope.go:117] "RemoveContainer" containerID="a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6" Sep 11 16:49:32.417270 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.417255 2553 scope.go:117] "RemoveContainer" containerID="a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6" Sep 11 16:49:32.417543 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:49:32.417527 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6\": container with ID starting with a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6 not found: ID does not exist" containerID="a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6" Sep 11 16:49:32.417597 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.417551 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6"} err="failed to get container status \"a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6\": rpc error: code = NotFound desc = could not find container \"a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6\": container with ID starting with a6f3ab957266f798368d1d6f2d75b201ee1d91e1f4c6496186e24513666a48e6 not found: ID does not exist" Sep 11 16:49:32.424790 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.424771 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm"] Sep 11 16:49:32.428650 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:32.428631 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-ea3c5-5d669fffd4-pqsgm"] Sep 11 16:49:33.398487 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:33.398458 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:34.076468 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:34.076436 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" path="/var/lib/kubelet/pods/bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a/volumes" Sep 11 16:49:36.184998 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.184911 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b"] Sep 11 16:49:36.185361 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.185159 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" containerID="cri-o://1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f" gracePeriod=30 Sep 11 16:49:36.206525 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.206498 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:36.305226 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.305179 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:36.401940 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.401905 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:36.501380 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.501346 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:36.565886 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.565851 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:36.776334 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:36.776246 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:49:38.402211 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:38.402172 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:43.397461 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:43.397422 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:48.397088 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:48.397048 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:48.397492 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:48.397152 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:49:53.397806 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:53.397757 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:58.397398 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:58.397360 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:49:58.777877 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:49:58.777844 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:50:01.856889 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.856852 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5"] Sep 11 16:50:01.857344 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.857152 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" Sep 11 16:50:01.857344 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.857162 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" Sep 11 16:50:01.857344 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.857218 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbaf5cf1-a97b-40f5-b77c-0c8365aa5e3a" containerName="sequence-graph-ea3c5" Sep 11 16:50:01.859923 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.859904 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:01.862491 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.862469 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-482a1-serving-cert\"" Sep 11 16:50:01.862599 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.862472 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-482a1-kube-rbac-proxy-sar-config\"" Sep 11 16:50:01.869407 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.869383 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5"] Sep 11 16:50:01.880346 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.880321 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:50:01.931103 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.931064 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-proxy-tls\") pod \"switch-graph-482a1-556896489f-gxlb5\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:01.931103 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:01.931111 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-openshift-service-ca-bundle\") pod \"switch-graph-482a1-556896489f-gxlb5\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:02.031633 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.031585 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-proxy-tls\") pod \"switch-graph-482a1-556896489f-gxlb5\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:02.031633 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.031639 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-openshift-service-ca-bundle\") pod \"switch-graph-482a1-556896489f-gxlb5\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:02.032312 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.032286 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-openshift-service-ca-bundle\") pod \"switch-graph-482a1-556896489f-gxlb5\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:02.034169 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.034151 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-proxy-tls\") pod \"switch-graph-482a1-556896489f-gxlb5\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:02.058385 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.058359 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:50:02.169955 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.169858 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:02.461293 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.461263 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5"] Sep 11 16:50:02.464157 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:50:02.464134 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8a1193_a8e5_4673_8e7a_7a7ca558b923.slice/crio-26b0d53406de2d0552c8a23573488daeb77c1fffe3c936a72b1c62d97f66c507 WatchSource:0}: Error finding container 26b0d53406de2d0552c8a23573488daeb77c1fffe3c936a72b1c62d97f66c507: Status 404 returned error can't find the container with id 26b0d53406de2d0552c8a23573488daeb77c1fffe3c936a72b1c62d97f66c507 Sep 11 16:50:02.518972 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:02.518947 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" event={"ID":"1b8a1193-a8e5-4673-8e7a-7a7ca558b923","Type":"ContainerStarted","Data":"26b0d53406de2d0552c8a23573488daeb77c1fffe3c936a72b1c62d97f66c507"} Sep 11 16:50:03.397430 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:03.397392 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:50:03.524616 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:03.524574 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" event={"ID":"1b8a1193-a8e5-4673-8e7a-7a7ca558b923","Type":"ContainerStarted","Data":"0e810827af6b3b55443f757aa7e048a39f0d0e66b91b7405e298807dd433c6ea"} Sep 11 16:50:03.524804 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:03.524726 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:03.541413 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:03.541364 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podStartSLOduration=2.541346333 podStartE2EDuration="2.541346333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:03.539853826 +0000 UTC m=+1959.954951370" watchObservedRunningTime="2026-09-11 16:50:03.541346333 +0000 UTC m=+1959.956443896" Sep 11 16:50:06.329792 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.329767 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:50:06.366857 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.366827 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-proxy-tls\") pod \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " Sep 11 16:50:06.367044 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.366932 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-openshift-service-ca-bundle\") pod \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\" (UID: \"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a\") " Sep 11 16:50:06.367298 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.367274 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" (UID: "109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:50:06.369117 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.369098 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" (UID: "109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:50:06.468354 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.468323 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:50:06.468354 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.468353 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:50:06.537649 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.537611 2553 generic.go:356] "Generic (PLEG): container finished" podID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerID="1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f" exitCode=0 Sep 11 16:50:06.537649 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.537649 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" Sep 11 16:50:06.537934 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.537716 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" event={"ID":"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a","Type":"ContainerDied","Data":"1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f"} Sep 11 16:50:06.537934 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.537773 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b" event={"ID":"109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a","Type":"ContainerDied","Data":"a079964b9187fdb3a28c7eddc7b7cb7d06b000a4025b165799b24d1381cabbf9"} Sep 11 16:50:06.537934 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.537806 2553 scope.go:117] "RemoveContainer" containerID="1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f" Sep 11 16:50:06.546718 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.546672 2553 scope.go:117] "RemoveContainer" containerID="1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f" Sep 11 16:50:06.546998 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:50:06.546979 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f\": container with ID starting with 1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f not found: ID does not exist" containerID="1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f" Sep 11 16:50:06.547068 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.547006 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f"} err="failed to get container status \"1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f\": rpc error: code = NotFound desc = could not find container \"1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f\": container with ID starting with 1c4d6b2f9328a3c893b79d8ff194132431c201d6fc234cfafd26ddc3cce4698f not found: ID does not exist" Sep 11 16:50:06.565148 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.565113 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b"] Sep 11 16:50:06.570703 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:06.570667 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-ddc34-98b445d6c-9zb8b"] Sep 11 16:50:08.076082 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:08.076051 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" path="/var/lib/kubelet/pods/109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a/volumes" Sep 11 16:50:09.534260 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:09.534235 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 16:50:36.412887 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.412853 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts"] Sep 11 16:50:36.413383 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.413222 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:50:36.413383 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.413306 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" Sep 11 16:50:36.413383 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.413326 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" Sep 11 16:50:36.413497 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.413402 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="109e6e3d-44c2-4115-9b7d-4bb6fc9ffb5a" containerName="splitter-graph-ddc34" Sep 11 16:50:36.416241 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.416227 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.418718 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.418694 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-20f8f-kube-rbac-proxy-sar-config\"" Sep 11 16:50:36.418830 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.418745 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-20f8f-serving-cert\"" Sep 11 16:50:36.423797 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.423778 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts"] Sep 11 16:50:36.511801 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.511771 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-proxy-tls\") pod \"splitter-graph-20f8f-7765c6fc54-q5zts\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.511977 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.511813 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-openshift-service-ca-bundle\") pod \"splitter-graph-20f8f-7765c6fc54-q5zts\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.574329 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.574283 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:50:36.612792 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.612759 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-proxy-tls\") pod \"splitter-graph-20f8f-7765c6fc54-q5zts\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.612959 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.612803 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-openshift-service-ca-bundle\") pod \"splitter-graph-20f8f-7765c6fc54-q5zts\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.613376 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.613359 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-openshift-service-ca-bundle\") pod \"splitter-graph-20f8f-7765c6fc54-q5zts\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.615432 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.615402 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-proxy-tls\") pod \"splitter-graph-20f8f-7765c6fc54-q5zts\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.727151 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.727104 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:36.994486 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:36.994403 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts"] Sep 11 16:50:36.996787 ip-10-0-132-119 kubenswrapper[2553]: W0911 16:50:36.996758 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d40ee3a_3454_445e_bc1d_3ed7f572adc1.slice/crio-399c62a77ad2fba9a05a3c71849ce1bf370f5ee23d3d0a6a5f8039b0f6ed126d WatchSource:0}: Error finding container 399c62a77ad2fba9a05a3c71849ce1bf370f5ee23d3d0a6a5f8039b0f6ed126d: Status 404 returned error can't find the container with id 399c62a77ad2fba9a05a3c71849ce1bf370f5ee23d3d0a6a5f8039b0f6ed126d Sep 11 16:50:37.655446 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:37.655411 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" event={"ID":"9d40ee3a-3454-445e-bc1d-3ed7f572adc1","Type":"ContainerStarted","Data":"3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff"} Sep 11 16:50:37.655864 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:37.655461 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" event={"ID":"9d40ee3a-3454-445e-bc1d-3ed7f572adc1","Type":"ContainerStarted","Data":"399c62a77ad2fba9a05a3c71849ce1bf370f5ee23d3d0a6a5f8039b0f6ed126d"} Sep 11 16:50:37.655864 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:37.655511 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:50:37.672837 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:37.672789 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podStartSLOduration=1.672776008 podStartE2EDuration="1.672776008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:50:37.671755167 +0000 UTC m=+1994.086852713" watchObservedRunningTime="2026-09-11 16:50:37.672776008 +0000 UTC m=+1994.087873550" Sep 11 16:50:43.664791 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:50:43.664767 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:52:31.893639 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:52:31.893603 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:55:07.568337 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:55:07.568305 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:55:23.840210 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:55:23.840176 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:58:50.990223 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:50.990191 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts"] Sep 11 16:58:50.992703 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:50.990402 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" containerID="cri-o://3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff" gracePeriod=30 Sep 11 16:58:51.034818 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:51.034777 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:58:51.144636 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:51.144589 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:58:51.206469 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:51.206441 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 16:58:53.663796 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:53.663756 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:58:58.663616 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:58:58.663567 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:59:03.664008 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:03.663962 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:59:03.664390 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:03.664069 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:59:08.663673 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:08.663632 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:59:13.663547 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:13.663497 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:59:18.663315 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:18.663226 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 16:59:21.131336 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.131310 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:59:21.190484 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.190451 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-proxy-tls\") pod \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " Sep 11 16:59:21.190668 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.190552 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-openshift-service-ca-bundle\") pod \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\" (UID: \"9d40ee3a-3454-445e-bc1d-3ed7f572adc1\") " Sep 11 16:59:21.190998 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.190969 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9d40ee3a-3454-445e-bc1d-3ed7f572adc1" (UID: "9d40ee3a-3454-445e-bc1d-3ed7f572adc1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 16:59:21.192625 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.192604 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9d40ee3a-3454-445e-bc1d-3ed7f572adc1" (UID: "9d40ee3a-3454-445e-bc1d-3ed7f572adc1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 16:59:21.291964 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.291870 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:59:21.291964 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.291904 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9d40ee3a-3454-445e-bc1d-3ed7f572adc1-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 16:59:21.611468 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.611374 2553 generic.go:356] "Generic (PLEG): container finished" podID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerID="3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff" exitCode=0 Sep 11 16:59:21.611468 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.611419 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" Sep 11 16:59:21.611468 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.611432 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" event={"ID":"9d40ee3a-3454-445e-bc1d-3ed7f572adc1","Type":"ContainerDied","Data":"3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff"} Sep 11 16:59:21.611762 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.611489 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts" event={"ID":"9d40ee3a-3454-445e-bc1d-3ed7f572adc1","Type":"ContainerDied","Data":"399c62a77ad2fba9a05a3c71849ce1bf370f5ee23d3d0a6a5f8039b0f6ed126d"} Sep 11 16:59:21.611762 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.611514 2553 scope.go:117] "RemoveContainer" containerID="3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff" Sep 11 16:59:21.620230 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.620202 2553 scope.go:117] "RemoveContainer" containerID="3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff" Sep 11 16:59:21.620493 ip-10-0-132-119 kubenswrapper[2553]: E0911 16:59:21.620476 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff\": container with ID starting with 3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff not found: ID does not exist" containerID="3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff" Sep 11 16:59:21.620548 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.620503 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff"} err="failed to get container status \"3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff\": rpc error: code = NotFound desc = could not find container \"3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff\": container with ID starting with 3795cce7c3f2e0b580714a04e73e7c85eea8a2f8e1650c2101149922a6618aff not found: ID does not exist" Sep 11 16:59:21.647456 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.647426 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts"] Sep 11 16:59:21.653707 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:21.653668 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-20f8f-7765c6fc54-q5zts"] Sep 11 16:59:22.076410 ip-10-0-132-119 kubenswrapper[2553]: I0911 16:59:22.076366 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" path="/var/lib/kubelet/pods/9d40ee3a-3454-445e-bc1d-3ed7f572adc1/volumes" Sep 11 17:01:15.474928 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:01:15.474893 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:01:19.919889 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:01:19.919848 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:21.026913 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:21.026880 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5"] Sep 11 17:06:21.029160 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:21.027109 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" containerID="cri-o://0e810827af6b3b55443f757aa7e048a39f0d0e66b91b7405e298807dd433c6ea" gracePeriod=30 Sep 11 17:06:21.073158 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:21.073116 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:22.230904 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.230865 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:22.518097 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.518018 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cqt7p/must-gather-qpknw"] Sep 11 17:06:22.518325 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.518314 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" Sep 11 17:06:22.518367 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.518327 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" Sep 11 17:06:22.518424 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.518385 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d40ee3a-3454-445e-bc1d-3ed7f572adc1" containerName="splitter-graph-20f8f" Sep 11 17:06:22.520867 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.520849 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.523882 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.523860 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cqt7p\"/\"kube-root-ca.crt\"" Sep 11 17:06:22.523997 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.523894 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cqt7p\"/\"openshift-service-ca.crt\"" Sep 11 17:06:22.532560 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.532539 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cqt7p/must-gather-qpknw"] Sep 11 17:06:22.546446 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.546412 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:22.646308 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.646272 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:22.647161 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.647142 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdqkq\" (UniqueName: \"kubernetes.io/projected/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-kube-api-access-zdqkq\") pod \"must-gather-qpknw\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.647247 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.647176 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-must-gather-output\") pod \"must-gather-qpknw\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.747710 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.747653 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdqkq\" (UniqueName: \"kubernetes.io/projected/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-kube-api-access-zdqkq\") pod \"must-gather-qpknw\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.747901 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.747727 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-must-gather-output\") pod \"must-gather-qpknw\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.748023 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.748001 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-must-gather-output\") pod \"must-gather-qpknw\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.755885 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.755861 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdqkq\" (UniqueName: \"kubernetes.io/projected/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-kube-api-access-zdqkq\") pod \"must-gather-qpknw\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:22.829513 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:22.829428 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:23.099400 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:23.099327 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cqt7p/must-gather-qpknw"] Sep 11 17:06:23.102089 ip-10-0-132-119 kubenswrapper[2553]: W0911 17:06:23.102057 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0ec802c_61ca_4bd7_9fab_d079e9396a2b.slice/crio-0ec269d60f9867ab364d905bebd8b2bb1e4f98fdc11f883c0f7fbafb6071b347 WatchSource:0}: Error finding container 0ec269d60f9867ab364d905bebd8b2bb1e4f98fdc11f883c0f7fbafb6071b347: Status 404 returned error can't find the container with id 0ec269d60f9867ab364d905bebd8b2bb1e4f98fdc11f883c0f7fbafb6071b347 Sep 11 17:06:23.103723 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:23.103708 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 11 17:06:23.156999 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:23.156964 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cqt7p/must-gather-qpknw" event={"ID":"d0ec802c-61ca-4bd7-9fab-d079e9396a2b","Type":"ContainerStarted","Data":"0ec269d60f9867ab364d905bebd8b2bb1e4f98fdc11f883c0f7fbafb6071b347"} Sep 11 17:06:24.534439 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:24.534253 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 17:06:29.533847 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:29.533802 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 17:06:30.190470 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:30.190418 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cqt7p/must-gather-qpknw" event={"ID":"d0ec802c-61ca-4bd7-9fab-d079e9396a2b","Type":"ContainerStarted","Data":"b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b"} Sep 11 17:06:30.190470 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:30.190477 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cqt7p/must-gather-qpknw" event={"ID":"d0ec802c-61ca-4bd7-9fab-d079e9396a2b","Type":"ContainerStarted","Data":"1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9"} Sep 11 17:06:30.208624 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:30.208575 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cqt7p/must-gather-qpknw" podStartSLOduration=2.146666201 podStartE2EDuration="8.208531735s" totalImagesPullingTime="6.061865534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:06:30.20699952 +0000 UTC m=+2946.622097059" watchObservedRunningTime="2026-09-11 17:06:30.208531735 +0000 UTC m=+2946.623629278" Sep 11 17:06:34.533155 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:34.533113 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 17:06:34.533515 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:34.533220 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 17:06:38.215299 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:38.215262 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:39.025145 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:39.025112 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:39.533093 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:39.533048 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 17:06:39.821258 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:39.821173 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:40.602954 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:40.602918 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:41.395005 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:41.394973 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:42.247568 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:42.247536 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:43.029045 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:43.029012 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:43.818914 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:43.818873 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:44.532469 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:44.532428 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 17:06:44.603403 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:44.603368 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:45.385393 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:45.385351 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:46.161769 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:46.161737 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:46.947388 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:46.947359 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:47.751227 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:47.751171 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:48.543119 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:48.543053 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-482a1-556896489f-gxlb5_1b8a1193-a8e5-4673-8e7a-7a7ca558b923/switch-graph-482a1/0.log" Sep 11 17:06:49.532446 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:49.532409 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 11 17:06:52.283618 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.283582 2553 generic.go:356] "Generic (PLEG): container finished" podID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerID="0e810827af6b3b55443f757aa7e048a39f0d0e66b91b7405e298807dd433c6ea" exitCode=0 Sep 11 17:06:52.283974 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.283627 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" event={"ID":"1b8a1193-a8e5-4673-8e7a-7a7ca558b923","Type":"ContainerDied","Data":"0e810827af6b3b55443f757aa7e048a39f0d0e66b91b7405e298807dd433c6ea"} Sep 11 17:06:52.399917 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.399887 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 17:06:52.483952 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.483921 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-proxy-tls\") pod \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " Sep 11 17:06:52.484134 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.483977 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-openshift-service-ca-bundle\") pod \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\" (UID: \"1b8a1193-a8e5-4673-8e7a-7a7ca558b923\") " Sep 11 17:06:52.484364 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.484341 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1b8a1193-a8e5-4673-8e7a-7a7ca558b923" (UID: "1b8a1193-a8e5-4673-8e7a-7a7ca558b923"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 11 17:06:52.486213 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.486194 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1b8a1193-a8e5-4673-8e7a-7a7ca558b923" (UID: "1b8a1193-a8e5-4673-8e7a-7a7ca558b923"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 11 17:06:52.585524 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.585498 2553 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-proxy-tls\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 17:06:52.585524 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:52.585525 2553 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b8a1193-a8e5-4673-8e7a-7a7ca558b923-openshift-service-ca-bundle\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 17:06:53.288964 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.288937 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" Sep 11 17:06:53.289417 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.288976 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5" event={"ID":"1b8a1193-a8e5-4673-8e7a-7a7ca558b923","Type":"ContainerDied","Data":"26b0d53406de2d0552c8a23573488daeb77c1fffe3c936a72b1c62d97f66c507"} Sep 11 17:06:53.289417 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.289038 2553 scope.go:117] "RemoveContainer" containerID="0e810827af6b3b55443f757aa7e048a39f0d0e66b91b7405e298807dd433c6ea" Sep 11 17:06:53.290906 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.290881 2553 generic.go:356] "Generic (PLEG): container finished" podID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerID="1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9" exitCode=0 Sep 11 17:06:53.291032 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.290914 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cqt7p/must-gather-qpknw" event={"ID":"d0ec802c-61ca-4bd7-9fab-d079e9396a2b","Type":"ContainerDied","Data":"1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9"} Sep 11 17:06:53.291195 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.291181 2553 scope.go:117] "RemoveContainer" containerID="1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9" Sep 11 17:06:53.318186 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.318158 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5"] Sep 11 17:06:53.321470 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.321452 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-482a1-556896489f-gxlb5"] Sep 11 17:06:53.567914 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:53.567829 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-cqt7p_must-gather-qpknw_d0ec802c-61ca-4bd7-9fab-d079e9396a2b/gather/0.log" Sep 11 17:06:54.007349 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:54.007318 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:54.077511 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:54.077481 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" path="/var/lib/kubelet/pods/1b8a1193-a8e5-4673-8e7a-7a7ca558b923/volumes" Sep 11 17:06:54.154776 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:54.154738 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:54.204442 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:54.204416 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:56.824796 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:56.824759 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xrwjl_b0c5ba56-2f15-44c5-bf94-d48bb6dd4f09/global-pull-secret-syncer/0.log" Sep 11 17:06:56.869857 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:56.869828 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" Sep 11 17:06:56.979456 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:56.979409 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-119.ec2.internal_1af1352b21b56122291aa11ed6868de7/haproxy/0.log" Sep 11 17:06:58.993612 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:58.993578 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-cqt7p/must-gather-qpknw"] Sep 11 17:06:58.994109 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:58.993807 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-cqt7p/must-gather-qpknw" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="copy" containerID="cri-o://b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b" gracePeriod=2 Sep 11 17:06:58.997978 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:58.997667 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-cqt7p/must-gather-qpknw"] Sep 11 17:06:59.224656 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.224634 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-cqt7p_must-gather-qpknw_d0ec802c-61ca-4bd7-9fab-d079e9396a2b/copy/0.log" Sep 11 17:06:59.226207 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.226191 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:59.228339 ip-10-0-132-119 kubenswrapper[2553]: E0911 17:06:59.228315 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qpknw\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-cqt7p\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:59.316259 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.316189 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-cqt7p_must-gather-qpknw_d0ec802c-61ca-4bd7-9fab-d079e9396a2b/copy/0.log" Sep 11 17:06:59.316552 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.316528 2553 generic.go:356] "Generic (PLEG): container finished" podID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerID="b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b" exitCode=143 Sep 11 17:06:59.316606 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.316580 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:59.316652 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.316624 2553 scope.go:117] "RemoveContainer" containerID="b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b" Sep 11 17:06:59.318821 ip-10-0-132-119 kubenswrapper[2553]: E0911 17:06:59.318795 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qpknw\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-cqt7p\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:06:59.323934 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.323918 2553 scope.go:117] "RemoveContainer" containerID="1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9" Sep 11 17:06:59.335984 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.335966 2553 scope.go:117] "RemoveContainer" containerID="b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b" Sep 11 17:06:59.336052 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.336017 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdqkq\" (UniqueName: \"kubernetes.io/projected/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-kube-api-access-zdqkq\") pod \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " Sep 11 17:06:59.336104 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.336087 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-must-gather-output\") pod \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\" (UID: \"d0ec802c-61ca-4bd7-9fab-d079e9396a2b\") " Sep 11 17:06:59.336289 ip-10-0-132-119 kubenswrapper[2553]: E0911 17:06:59.336261 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b\": container with ID starting with b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b not found: ID does not exist" containerID="b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b" Sep 11 17:06:59.336347 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.336301 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b"} err="failed to get container status \"b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b\": rpc error: code = NotFound desc = could not find container \"b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b\": container with ID starting with b701791554c3bd500775864c3207f3bae6ec5c985f46102e9f697c966fe6882b not found: ID does not exist" Sep 11 17:06:59.336347 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.336327 2553 scope.go:117] "RemoveContainer" containerID="1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9" Sep 11 17:06:59.336828 ip-10-0-132-119 kubenswrapper[2553]: E0911 17:06:59.336674 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9\": container with ID starting with 1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9 not found: ID does not exist" containerID="1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9" Sep 11 17:06:59.336893 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.336837 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9"} err="failed to get container status \"1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9\": rpc error: code = NotFound desc = could not find container \"1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9\": container with ID starting with 1977a78da976b97f10d5d38acaa3fd0c333059358e80d8b6701e51eef9eb4ac9 not found: ID does not exist" Sep 11 17:06:59.337501 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.337481 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "d0ec802c-61ca-4bd7-9fab-d079e9396a2b" (UID: "d0ec802c-61ca-4bd7-9fab-d079e9396a2b"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 11 17:06:59.338241 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.338224 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-kube-api-access-zdqkq" (OuterVolumeSpecName: "kube-api-access-zdqkq") pod "d0ec802c-61ca-4bd7-9fab-d079e9396a2b" (UID: "d0ec802c-61ca-4bd7-9fab-d079e9396a2b"). InnerVolumeSpecName "kube-api-access-zdqkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 11 17:06:59.436778 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.436748 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdqkq\" (UniqueName: \"kubernetes.io/projected/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-kube-api-access-zdqkq\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 17:06:59.436778 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.436772 2553 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d0ec802c-61ca-4bd7-9fab-d079e9396a2b-must-gather-output\") on node \"ip-10-0-132-119.ec2.internal\" DevicePath \"\"" Sep 11 17:06:59.531628 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.531596 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:59.592108 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:06:59.592018 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6b2pn_a2a9f964-4217-4878-8c24-b086a1202c5d/konnectivity-agent/0.log" limit=1024 Sep 11 17:06:59.627361 ip-10-0-132-119 kubenswrapper[2553]: E0911 17:06:59.627328 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-qpknw\" is forbidden: User \"system:node:ip-10-0-132-119.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-cqt7p\": no relationship found between node 'ip-10-0-132-119.ec2.internal' and this object" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" pod="openshift-must-gather-cqt7p/must-gather-qpknw" Sep 11 17:07:00.076247 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:00.076213 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" path="/var/lib/kubelet/pods/d0ec802c-61ca-4bd7-9fab-d079e9396a2b/volumes" Sep 11 17:07:01.599398 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.599363 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-c994r_43c3f62e-6fd8-4bd2-8082-8750a79ec05c/kube-state-metrics/0.log" Sep 11 17:07:01.619836 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.619806 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-c994r_43c3f62e-6fd8-4bd2-8082-8750a79ec05c/kube-rbac-proxy-main/0.log" Sep 11 17:07:01.637441 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.637410 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-c994r_43c3f62e-6fd8-4bd2-8082-8750a79ec05c/kube-rbac-proxy-self/0.log" Sep 11 17:07:01.688928 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.688897 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-vp8hv_c80e75b2-05fd-4328-b748-4bdeca6242c7/monitoring-plugin/0.log" Sep 11 17:07:01.716528 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.716442 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hjg5p_cb941391-2259-4a87-8a26-e679bc8ef60f/node-exporter/0.log" Sep 11 17:07:01.733566 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.733542 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hjg5p_cb941391-2259-4a87-8a26-e679bc8ef60f/kube-rbac-proxy/0.log" Sep 11 17:07:01.752332 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:01.752307 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hjg5p_cb941391-2259-4a87-8a26-e679bc8ef60f/init-textfile/0.log" Sep 11 17:07:02.023350 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.023265 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/prometheus/0.log" Sep 11 17:07:02.038541 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.038514 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/config-reloader/0.log" Sep 11 17:07:02.056380 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.056354 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/thanos-sidecar/0.log" Sep 11 17:07:02.072845 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.072821 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/kube-rbac-proxy-web/0.log" Sep 11 17:07:02.088806 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.088779 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/kube-rbac-proxy/0.log" Sep 11 17:07:02.110164 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.110141 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/kube-rbac-proxy-thanos/0.log" Sep 11 17:07:02.128509 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.128490 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_abc53d86-47e4-488e-a617-2205414a2a40/init-config-reloader/0.log" Sep 11 17:07:02.156809 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.156788 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-7glw9_2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa/prometheus-operator/0.log" Sep 11 17:07:02.169151 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.169131 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-7glw9_2e2edf7d-7c11-4d5f-9ee5-4a7157bc7faa/kube-rbac-proxy/0.log" Sep 11 17:07:02.292766 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.292652 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ccddbcc9c-dsvp7_d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b/thanos-query/0.log" Sep 11 17:07:02.310808 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.310781 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ccddbcc9c-dsvp7_d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b/kube-rbac-proxy-web/0.log" Sep 11 17:07:02.337274 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.337251 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ccddbcc9c-dsvp7_d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b/kube-rbac-proxy/0.log" Sep 11 17:07:02.356351 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.356332 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ccddbcc9c-dsvp7_d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b/prom-label-proxy/0.log" Sep 11 17:07:02.383931 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.383911 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ccddbcc9c-dsvp7_d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b/kube-rbac-proxy-rules/0.log" Sep 11 17:07:02.402940 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:02.402914 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-ccddbcc9c-dsvp7_d3c6f21f-5b5f-470d-90ea-cdd5aa27bd9b/kube-rbac-proxy-metrics/0.log" Sep 11 17:07:04.476079 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:04.476041 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-7mr9t_9e992fc4-f472-48bc-b9fa-bccf95275e45/download-server/0.log" Sep 11 17:07:05.023229 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023199 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n"] Sep 11 17:07:05.023495 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023483 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="copy" Sep 11 17:07:05.023543 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023497 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="copy" Sep 11 17:07:05.023543 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023506 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="gather" Sep 11 17:07:05.023543 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023513 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="gather" Sep 11 17:07:05.023543 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023521 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" Sep 11 17:07:05.023543 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023526 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" Sep 11 17:07:05.023715 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023580 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b8a1193-a8e5-4673-8e7a-7a7ca558b923" containerName="switch-graph-482a1" Sep 11 17:07:05.023715 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023588 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="copy" Sep 11 17:07:05.023715 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.023597 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ec802c-61ca-4bd7-9fab-d079e9396a2b" containerName="gather" Sep 11 17:07:05.025400 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.025381 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.028170 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.028151 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8n4t\"/\"kube-root-ca.crt\"" Sep 11 17:07:05.029222 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.029200 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g8n4t\"/\"default-dockercfg-rjl6t\"" Sep 11 17:07:05.029317 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.029225 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8n4t\"/\"openshift-service-ca.crt\"" Sep 11 17:07:05.033962 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.033938 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n"] Sep 11 17:07:05.081591 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.081562 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-lib-modules\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.081786 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.081613 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmlr4\" (UniqueName: \"kubernetes.io/projected/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-kube-api-access-nmlr4\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.081786 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.081651 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-sys\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.081786 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.081704 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-proc\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.081786 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.081726 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-podres\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.182923 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.182882 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-lib-modules\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183110 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.182938 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmlr4\" (UniqueName: \"kubernetes.io/projected/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-kube-api-access-nmlr4\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183110 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.182975 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-sys\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183110 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.182993 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-proc\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183110 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.183014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-podres\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183110 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.183085 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-sys\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183110 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.183110 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-podres\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183347 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.183084 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-lib-modules\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.183347 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.183117 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-proc\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.191138 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.191102 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmlr4\" (UniqueName: \"kubernetes.io/projected/dfb6ebaf-e3bd-4934-b270-dba61d43b9e6-kube-api-access-nmlr4\") pod \"perf-node-gather-daemonset-sgb4n\" (UID: \"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.335274 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.335176 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:05.602863 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.602801 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n"] Sep 11 17:07:05.605356 ip-10-0-132-119 kubenswrapper[2553]: W0911 17:07:05.605327 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddfb6ebaf_e3bd_4934_b270_dba61d43b9e6.slice/crio-69acaf0959bd78b06bfe11ce64fa682a2a8dec9757deb4d3e54d11cc954eb4f7 WatchSource:0}: Error finding container 69acaf0959bd78b06bfe11ce64fa682a2a8dec9757deb4d3e54d11cc954eb4f7: Status 404 returned error can't find the container with id 69acaf0959bd78b06bfe11ce64fa682a2a8dec9757deb4d3e54d11cc954eb4f7 Sep 11 17:07:05.851699 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.851655 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-72pf4_09bf8587-6328-4fd6-8421-9b612669bc63/dns/0.log" Sep 11 17:07:05.867119 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.867042 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-72pf4_09bf8587-6328-4fd6-8421-9b612669bc63/kube-rbac-proxy/0.log" Sep 11 17:07:05.922347 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:05.922316 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bqpcw_0475e5f9-fcb0-4165-b925-ca7901a3b53d/dns-node-resolver/0.log" Sep 11 17:07:06.343581 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:06.343548 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" event={"ID":"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6","Type":"ContainerStarted","Data":"79d4a7fbdfc1bf3401bc8a539f7c7ba3deafee28645a22ae982891d50f556514"} Sep 11 17:07:06.343795 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:06.343596 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" event={"ID":"dfb6ebaf-e3bd-4934-b270-dba61d43b9e6","Type":"ContainerStarted","Data":"69acaf0959bd78b06bfe11ce64fa682a2a8dec9757deb4d3e54d11cc954eb4f7"} Sep 11 17:07:06.343795 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:06.343665 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:06.360052 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:06.360006 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" podStartSLOduration=1.3599910579999999 podStartE2EDuration="1.359991058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:07:06.359541473 +0000 UTC m=+2982.774639018" watchObservedRunningTime="2026-09-11 17:07:06.359991058 +0000 UTC m=+2982.775088601" Sep 11 17:07:06.425594 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:06.425565 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-62tgl_1a762e41-b42c-470f-bd77-4d790aed8532/node-ca/0.log" Sep 11 17:07:07.460467 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:07.460432 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r7z8b_a167ebfd-9f52-46b9-b6c8-367d461a31d6/serve-healthcheck-canary/0.log" Sep 11 17:07:07.901100 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:07.901016 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7fwg_6e4169a0-5b02-4c7b-912c-b3f944d03f43/kube-rbac-proxy/0.log" Sep 11 17:07:07.916194 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:07.916162 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7fwg_6e4169a0-5b02-4c7b-912c-b3f944d03f43/exporter/0.log" Sep 11 17:07:07.932096 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:07.932072 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7fwg_6e4169a0-5b02-4c7b-912c-b3f944d03f43/extractor/0.log" Sep 11 17:07:12.356951 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:12.356924 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-sgb4n" Sep 11 17:07:15.272927 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.272893 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/kube-multus-additional-cni-plugins/0.log" Sep 11 17:07:15.292131 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.292103 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/egress-router-binary-copy/0.log" Sep 11 17:07:15.309573 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.309543 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/cni-plugins/0.log" Sep 11 17:07:15.325644 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.325621 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/bond-cni-plugin/0.log" Sep 11 17:07:15.340880 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.340858 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/routeoverride-cni/0.log" Sep 11 17:07:15.356026 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.356001 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/whereabouts-cni-bincopy/0.log" Sep 11 17:07:15.372973 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.372950 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6g24b_a94516ff-ab02-4f35-82d4-6ac996fc5cb2/whereabouts-cni/0.log" Sep 11 17:07:15.579101 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.579025 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tzp4p_a126d30a-b32c-4f2a-9bab-fc887339dddb/kube-multus/0.log" Sep 11 17:07:15.596397 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.596367 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5wxnc_4ac947c7-0e15-4a76-88b8-0004ce8933e7/network-metrics-daemon/0.log" Sep 11 17:07:15.611299 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:15.611268 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5wxnc_4ac947c7-0e15-4a76-88b8-0004ce8933e7/kube-rbac-proxy/0.log" Sep 11 17:07:16.652199 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.652107 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/ovn-controller/0.log" Sep 11 17:07:16.698035 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.697996 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/ovn-acl-logging/0.log" Sep 11 17:07:16.717099 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.717060 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/kube-rbac-proxy-node/0.log" Sep 11 17:07:16.733381 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.733346 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/kube-rbac-proxy-ovn-metrics/0.log" Sep 11 17:07:16.747928 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.747898 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/northd/0.log" Sep 11 17:07:16.763178 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.763146 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/nbdb/0.log" Sep 11 17:07:16.779556 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.779521 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/sbdb/0.log" Sep 11 17:07:16.829002 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:16.828975 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bdf94_6cb0b991-2dce-4a62-b6a6-30e8dd32023d/ovnkube-controller/0.log" Sep 11 17:07:17.890201 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:17.890174 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-j7pkn_172fbab1-39a1-4a58-bdab-94faf02f1526/network-check-target-container/0.log" Sep 11 17:07:18.702366 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:18.702342 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-gx7gz_c3484cd6-fa9f-4e35-bfb7-15d16130f884/iptables-alerter/0.log" Sep 11 17:07:19.292914 ip-10-0-132-119 kubenswrapper[2553]: I0911 17:07:19.292889 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-86frm_c9efec09-dc0f-41b8-8dfd-692abdc6d11c/tuned/0.log"