Sep 18 16:49:15.185957 ip-10-0-135-188 systemd[1]: Starting Kubernetes Kubelet... Sep 18 16:49:18.713843 ip-10-0-135-188 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 16:49:18.713843 ip-10-0-135-188 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 18 16:49:18.713843 ip-10-0-135-188 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 16:49:18.713843 ip-10-0-135-188 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 18 16:49:18.713843 ip-10-0-135-188 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 18 16:49:18.714594 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.713907 2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 18 16:49:18.717108 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717086 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 16:49:18.717108 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717106 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717111 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717115 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717118 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717121 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717124 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717127 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717131 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717135 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717138 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717149 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717153 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717157 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717160 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717163 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717166 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717168 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717171 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717174 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717176 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 16:49:18.717184 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717179 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717182 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717184 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717187 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717189 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717193 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717196 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717198 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717201 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717204 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717206 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717209 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717212 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717215 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717218 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717220 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717222 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717226 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717230 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717232 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 16:49:18.717703 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717235 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717238 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717242 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717245 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717247 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717250 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717254 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717256 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717258 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717261 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717264 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717266 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717268 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717271 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717275 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717277 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717280 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717283 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717285 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717288 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 16:49:18.718229 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717290 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717292 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717295 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717297 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717300 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717302 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717305 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717308 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717310 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717313 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717315 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717319 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717321 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717324 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717326 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717329 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717331 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717335 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717339 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 16:49:18.718743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717341 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717344 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717348 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717351 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717353 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717356 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717358 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717362 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717365 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717367 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717370 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717372 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717375 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717377 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717380 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717383 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717385 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717388 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.717390 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718111 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 16:49:18.719219 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718118 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718121 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718123 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718126 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718129 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718132 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718134 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718139 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718143 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718146 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718149 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718152 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718157 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718161 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718164 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718167 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718171 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718174 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718178 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 16:49:18.719777 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718180 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718183 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718186 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718189 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718192 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718195 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718197 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718200 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718203 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718205 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718208 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718211 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718213 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718215 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718218 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718220 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718223 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718226 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718228 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718230 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 16:49:18.720290 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718233 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718236 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718239 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718242 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718244 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718247 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718250 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718253 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718255 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718259 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718262 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718265 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718268 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718271 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718274 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718276 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718279 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718281 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718284 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718286 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 16:49:18.720815 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718289 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718291 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718293 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718296 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718299 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718301 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718303 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718306 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718308 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718311 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718314 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718317 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718319 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718323 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718325 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718328 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718330 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718333 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718336 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718339 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 16:49:18.721320 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718342 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718344 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718347 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718350 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718353 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718355 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718358 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718360 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718363 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718365 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718368 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718370 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718373 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718376 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718378 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718381 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718383 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718386 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718388 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718812 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 16:49:18.722069 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718817 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718820 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718823 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718826 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718829 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718832 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718835 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718837 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718840 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718843 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718846 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718850 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718852 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718855 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718858 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718861 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718863 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718866 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718870 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 16:49:18.722889 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718874 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718877 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718880 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718883 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718885 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718888 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718890 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718893 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718896 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718899 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718901 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718904 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718906 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718909 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718912 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718914 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718917 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718919 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718922 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718925 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 16:49:18.723382 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718927 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718930 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718932 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718936 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718939 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718942 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718944 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718947 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718950 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718952 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718955 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718957 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718960 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718962 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718965 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718967 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718970 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718972 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718975 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718977 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 16:49:18.723918 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718980 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718982 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718985 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718987 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718990 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718992 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718995 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.718997 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719000 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719002 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719004 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719007 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719010 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719012 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719015 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719017 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719020 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719023 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719026 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719028 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 16:49:18.724421 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719031 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719034 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719036 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719039 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719041 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719046 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719049 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719053 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719057 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719060 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719062 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719066 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719068 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719071 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719073 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719076 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719079 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719081 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719083 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 16:49:18.724946 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719162 2586 flags.go:64] FLAG: --address="0.0.0.0" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719200 2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719207 2586 flags.go:64] FLAG: --anonymous-auth="true" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719211 2586 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719215 2586 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719219 2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719224 2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719228 2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719232 2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719235 2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719238 2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719242 2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719248 2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719251 2586 flags.go:64] FLAG: --cgroup-root="" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719254 2586 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719257 2586 flags.go:64] FLAG: --client-ca-file="" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719260 2586 flags.go:64] FLAG: --cloud-provider="external" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719263 2586 flags.go:64] FLAG: --cluster-dns="[]" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719267 2586 flags.go:64] FLAG: --cluster-domain="" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719270 2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719273 2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719276 2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719279 2586 flags.go:64] FLAG: --container-log-max-files="5" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719283 2586 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 18 16:49:18.725414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719286 2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719289 2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719293 2586 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719296 2586 flags.go:64] FLAG: --contention-profiling="false" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719299 2586 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719301 2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719305 2586 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719307 2586 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719311 2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719314 2586 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719317 2586 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719320 2586 flags.go:64] FLAG: --enable-load-reader="false" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719322 2586 flags.go:64] FLAG: --enable-server="true" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719325 2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719331 2586 flags.go:64] FLAG: --event-burst="100" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719334 2586 flags.go:64] FLAG: --event-qps="50" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719337 2586 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719340 2586 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719343 2586 flags.go:64] FLAG: --eviction-hard="" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719347 2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719351 2586 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719355 2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719358 2586 flags.go:64] FLAG: --eviction-soft="" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719362 2586 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719364 2586 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 18 16:49:18.726002 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719367 2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719370 2586 flags.go:64] FLAG: --experimental-mounter-path="" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719373 2586 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719376 2586 flags.go:64] FLAG: --fail-swap-on="true" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719379 2586 flags.go:64] FLAG: --feature-gates="" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719383 2586 flags.go:64] FLAG: --file-check-frequency="20s" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719386 2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719389 2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719392 2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719395 2586 flags.go:64] FLAG: --healthz-port="10248" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719398 2586 flags.go:64] FLAG: --help="false" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719402 2586 flags.go:64] FLAG: --hostname-override="ip-10-0-135-188.ec2.internal" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719405 2586 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719408 2586 flags.go:64] FLAG: --http-check-frequency="20s" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719411 2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719414 2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719417 2586 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719420 2586 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719422 2586 flags.go:64] FLAG: --image-service-endpoint="" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719425 2586 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719428 2586 flags.go:64] FLAG: --kube-api-burst="100" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719431 2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719434 2586 flags.go:64] FLAG: --kube-api-qps="50" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719438 2586 flags.go:64] FLAG: --kube-reserved="" Sep 18 16:49:18.726716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719441 2586 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719444 2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719447 2586 flags.go:64] FLAG: --kubelet-cgroups="" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719452 2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719455 2586 flags.go:64] FLAG: --lock-file="" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719459 2586 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719462 2586 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719465 2586 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719470 2586 flags.go:64] FLAG: --log-json-split-stream="false" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719473 2586 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719476 2586 flags.go:64] FLAG: --log-text-split-stream="false" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719478 2586 flags.go:64] FLAG: --logging-format="text" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719481 2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719485 2586 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719487 2586 flags.go:64] FLAG: --manifest-url="" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719490 2586 flags.go:64] FLAG: --manifest-url-header="" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719495 2586 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719498 2586 flags.go:64] FLAG: --max-open-files="1000000" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719502 2586 flags.go:64] FLAG: --max-pods="110" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719505 2586 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719507 2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719510 2586 flags.go:64] FLAG: --memory-manager-policy="None" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719513 2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719516 2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719519 2586 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 18 16:49:18.727309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719522 2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719530 2586 flags.go:64] FLAG: --node-status-max-images="50" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719532 2586 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719535 2586 flags.go:64] FLAG: --oom-score-adj="-999" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719538 2586 flags.go:64] FLAG: --pod-cidr="" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719541 2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719547 2586 flags.go:64] FLAG: --pod-manifest-path="" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719566 2586 flags.go:64] FLAG: --pod-max-pids="-1" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719569 2586 flags.go:64] FLAG: --pods-per-core="0" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719572 2586 flags.go:64] FLAG: --port="10250" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719577 2586 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719580 2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00fd0dccfbe4ab4d9" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719583 2586 flags.go:64] FLAG: --qos-reserved="" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719586 2586 flags.go:64] FLAG: --read-only-port="10255" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719590 2586 flags.go:64] FLAG: --register-node="true" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719593 2586 flags.go:64] FLAG: --register-with-taints="" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719597 2586 flags.go:64] FLAG: --registry-burst="10" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719600 2586 flags.go:64] FLAG: --registry-qps="5" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719603 2586 flags.go:64] FLAG: --reserved-cpus="" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719605 2586 flags.go:64] FLAG: --reserved-memory="" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719609 2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719612 2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719615 2586 flags.go:64] FLAG: --rotate-certificates="false" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719618 2586 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719621 2586 flags.go:64] FLAG: --runonce="false" Sep 18 16:49:18.727959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719623 2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719627 2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719630 2586 flags.go:64] FLAG: --seccomp-default="false" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719632 2586 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719635 2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719638 2586 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719641 2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719645 2586 flags.go:64] FLAG: --storage-driver-password="root" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719647 2586 flags.go:64] FLAG: --storage-driver-secure="false" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719650 2586 flags.go:64] FLAG: --storage-driver-table="stats" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719653 2586 flags.go:64] FLAG: --storage-driver-user="root" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719656 2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719659 2586 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719661 2586 flags.go:64] FLAG: --system-cgroups="" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719667 2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719673 2586 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719676 2586 flags.go:64] FLAG: --tls-cert-file="" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719680 2586 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719684 2586 flags.go:64] FLAG: --tls-min-version="" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719687 2586 flags.go:64] FLAG: --tls-private-key-file="" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719690 2586 flags.go:64] FLAG: --topology-manager-policy="none" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719693 2586 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719696 2586 flags.go:64] FLAG: --topology-manager-scope="container" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719699 2586 flags.go:64] FLAG: --v="2" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719703 2586 flags.go:64] FLAG: --version="false" Sep 18 16:49:18.728632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719707 2586 flags.go:64] FLAG: --vmodule="" Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719711 2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.719714 2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719839 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719844 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719847 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719850 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719853 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719855 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719858 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719861 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719863 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719866 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719869 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719871 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719874 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719877 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719879 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719882 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719885 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 16:49:18.729264 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719887 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719892 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719894 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719897 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719901 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719904 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719906 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719909 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719912 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719915 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719917 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719920 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719922 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719926 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719928 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719931 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719934 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719936 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719939 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719942 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 16:49:18.729789 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719944 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719947 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719949 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719952 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719954 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719957 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719960 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719962 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719965 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719968 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719970 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719973 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719975 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719979 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719982 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719985 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719990 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719992 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719995 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.719997 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 16:49:18.730289 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720000 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720002 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720006 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720009 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720011 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720014 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720017 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720020 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720022 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720025 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720028 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720030 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720035 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720038 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720041 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720044 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720047 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720050 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720052 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 16:49:18.730803 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720055 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720058 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720060 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720064 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720068 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720071 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720075 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720079 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720082 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720086 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720089 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720092 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720094 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720097 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720100 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720103 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720106 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720109 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720112 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720115 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 16:49:18.731285 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720117 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720121 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.720125 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.720130 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.727941 2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.727962 2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.727983 2586 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.727989 2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728047 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728053 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728057 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728060 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728064 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728067 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 16:49:18.731821 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728069 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728072 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728075 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728077 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728081 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728084 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728086 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728089 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728092 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728096 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728100 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728103 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728106 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728110 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728114 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728117 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728120 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728123 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728126 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 16:49:18.732240 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728129 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728134 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728137 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728140 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728144 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728148 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728151 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728154 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728157 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728159 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728162 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728165 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728167 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728170 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728173 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728175 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728178 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728180 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728183 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 16:49:18.732729 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728186 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728188 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728191 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728194 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728198 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728200 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728203 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728206 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728208 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728211 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728214 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728217 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728219 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728222 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728224 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728227 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728230 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728233 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728235 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728238 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 16:49:18.733262 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728241 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728243 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728246 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728249 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728251 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728254 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728257 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728259 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728262 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728264 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728267 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728269 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728272 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728275 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728278 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728281 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728284 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728287 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728289 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728292 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 16:49:18.733808 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728295 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728298 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728300 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728303 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728305 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728308 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728310 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728313 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728316 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728318 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728321 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728323 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728326 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728329 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728332 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 16:49:18.734323 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.728337 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728453 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728459 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728462 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728465 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728468 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728471 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728474 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728476 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728479 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728482 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728484 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728487 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728490 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728493 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728496 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728499 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728502 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728505 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728508 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 18 16:49:18.734727 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728510 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728513 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728516 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728519 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728521 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728524 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728526 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728529 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728532 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728535 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728539 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728541 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728544 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728547 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728550 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728572 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728576 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728581 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728585 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728589 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 18 16:49:18.735294 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728592 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728595 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728597 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728601 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728605 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728608 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728612 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728615 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728618 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728621 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728624 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728627 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728629 2586 feature_gate.go:328] unrecognized feature gate: Example Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728632 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728635 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728638 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728641 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728644 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728647 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 18 16:49:18.735825 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728650 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728653 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728656 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728658 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728661 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728664 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728667 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728669 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728671 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728674 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728676 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728679 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728681 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728684 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728687 2586 feature_gate.go:328] unrecognized feature gate: Example2 Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728689 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728692 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728695 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728697 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728699 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 18 16:49:18.736310 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728709 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728712 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728715 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728717 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728720 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728722 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728725 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728727 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728731 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728734 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728737 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728760 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728765 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728768 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728771 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728774 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728781 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728784 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728787 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 18 16:49:18.736865 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728789 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:18.728792 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.728797 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.728936 2586 server.go:962] "Client rotation is on, will bootstrap in background" Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.731845 2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.732832 2586 server.go:1019] "Starting client certificate rotation" Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.732940 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.732974 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.735200 2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 16:49:18.737348 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.736409 2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 18 16:49:18.742480 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.742455 2586 log.go:25] "Validated CRI v1 runtime API" Sep 18 16:49:18.744458 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.744441 2586 log.go:25] "Validated CRI v1 image API" Sep 18 16:49:18.745332 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.745314 2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 18 16:49:18.748388 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.748361 2586 fs.go:135] Filesystem UUIDs: map[341a6a4c-ab85-417c-a11d-2963ef24b545:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a39ba823-4ff9-4581-a2e5-8db7be0e8d1f:/dev/nvme0n1p3] Sep 18 16:49:18.748388 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.748384 2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 18 16:49:18.753587 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.753442 2586 manager.go:216] Machine: {Timestamp:2026-09-18 16:49:18.752943567 +0000 UTC m=+2.029068751 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098989 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25ed12df328c6ccc080be28ccfdb7f SystemUUID:ec25ed12-df32-8c6c-cc08-0be28ccfdb7f BootID:07e92471-b514-471d-a2b3-54bd6a6b4c68 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c0:95:6b:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c0:95:6b:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:24:e2:fa:1a:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 18 16:49:18.753587 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.753576 2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 18 16:49:18.753749 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.753669 2586 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 18 16:49:18.755295 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.755257 2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 18 16:49:18.755438 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.755296 2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 18 16:49:18.756461 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.756446 2586 topology_manager.go:138] "Creating topology manager with none policy" Sep 18 16:49:18.756461 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.756461 2586 container_manager_linux.go:309] "Creating device plugin manager" Sep 18 16:49:18.756570 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.756469 2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 16:49:18.756570 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.756482 2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 18 16:49:18.756643 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.756581 2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 18 16:49:18.760422 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760405 2586 state_mem.go:36] "Initialized new in-memory state store" Sep 18 16:49:18.760633 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760621 2586 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 18 16:49:18.760677 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760660 2586 kubelet.go:486] "Attempting to sync node with API server" Sep 18 16:49:18.760677 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760670 2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 18 16:49:18.760743 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760682 2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 18 16:49:18.760743 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760705 2586 kubelet.go:398] "Adding apiserver pod source" Sep 18 16:49:18.760743 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.760714 2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 18 16:49:18.761120 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.761108 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 16:49:18.761152 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.761128 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 18 16:49:18.763839 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.763820 2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 18 16:49:18.764895 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.764873 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 16:49:18.766302 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.766286 2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 18 16:49:18.766364 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.766317 2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 18 16:49:18.767496 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767481 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 18 16:49:18.767496 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767497 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767503 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767509 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767517 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767523 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767537 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767542 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767565 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767587 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767604 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 18 16:49:18.767624 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767610 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 18 16:49:18.768013 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767636 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 18 16:49:18.768013 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.767642 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 18 16:49:18.771214 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.771190 2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 18 16:49:18.771313 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.771262 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 16:49:18.771369 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.771353 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 18 16:49:18.771882 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.771860 2586 server.go:1268] "Started kubelet" Sep 18 16:49:18.771988 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.771936 2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 18 16:49:18.772033 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.771953 2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 18 16:49:18.772033 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.772020 2586 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 18 16:49:18.772908 ip-10-0-135-188 systemd[1]: Started Kubernetes Kubelet. Sep 18 16:49:18.773354 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.773235 2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 18 16:49:18.774010 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.773992 2586 server.go:310] "Adding debug handlers to kubelet server" Sep 18 16:49:18.776194 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.776174 2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 18 16:49:18.781052 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.781027 2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 18 16:49:18.781052 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.781046 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 18 16:49:18.781277 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.781258 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:18.806292 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.806170 2586 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 18 16:49:18.806439 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.806311 2586 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 18 16:49:18.806439 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.806157 2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 18 16:49:18.806849 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.806829 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7cr2m" Sep 18 16:49:18.807116 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807102 2586 reconstruct.go:97] "Volume reconstruction finished" Sep 18 16:49:18.807221 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807186 2586 reconciler.go:29] "Reconciler: start to sync state" Sep 18 16:49:18.807340 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807234 2586 factory.go:55] Registering systemd factory Sep 18 16:49:18.807340 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807253 2586 factory.go:223] Registration of the systemd container factory successfully Sep 18 16:49:18.807504 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.807478 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 18 16:49:18.807765 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.807742 2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 18 16:49:18.807851 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807785 2586 factory.go:153] Registering CRI-O factory Sep 18 16:49:18.807851 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807800 2586 factory.go:223] Registration of the crio container factory successfully Sep 18 16:49:18.807936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807867 2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 18 16:49:18.807936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807887 2586 factory.go:103] Registering Raw factory Sep 18 16:49:18.807936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.807905 2586 manager.go:1196] Started watching for new ooms in manager Sep 18 16:49:18.808841 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.808813 2586 manager.go:349] Starting recovery of all containers Sep 18 16:49:18.814901 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.814872 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7cr2m" Sep 18 16:49:18.819671 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.819649 2586 manager.go:354] Recovery completed Sep 18 16:49:18.821826 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.820471 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-188.ec2.internal.18d67849724b32b8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-188.ec2.internal,UID:ip-10-0-135-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-188.ec2.internal,},FirstTimestamp:2026-09-18 16:49:18.771827384 +0000 UTC m=+2.047952548,LastTimestamp:2026-09-18 16:49:18.771827384 +0000 UTC m=+2.047952548,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-188.ec2.internal,}" Sep 18 16:49:18.824460 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.824444 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:18.828085 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828062 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:18.828157 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828098 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:18.828157 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828110 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:18.828738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828723 2586 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 18 16:49:18.828738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828736 2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 18 16:49:18.828852 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828752 2586 state_mem.go:36] "Initialized new in-memory state store" Sep 18 16:49:18.828852 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.828768 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:18.829879 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.829863 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:18.829927 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.829891 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:18.829927 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.829901 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:18.831121 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.831105 2586 policy_none.go:49] "None policy: Start" Sep 18 16:49:18.831172 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.831129 2586 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 18 16:49:18.831172 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.831142 2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 18 16:49:18.833588 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.833571 2586 policy_none.go:47] "Start" Sep 18 16:49:18.871254 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871235 2586 manager.go:337] "Starting Device Plugin manager" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.871277 2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871288 2586 server.go:87] "Starting device plugin registration server" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871542 2586 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871587 2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871744 2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871821 2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.871828 2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.872592 2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 18 16:49:18.887990 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.872656 2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:18.908337 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.908304 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 18 16:49:18.909713 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.909691 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 18 16:49:18.909820 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.909764 2586 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 18 16:49:18.909820 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.909800 2586 kubelet.go:2448] "Starting kubelet main sync loop" Sep 18 16:49:18.909917 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.909851 2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 18 16:49:18.911803 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.911779 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 16:49:18.972171 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.972075 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:18.973271 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.973249 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:18.973376 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.973284 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:18.973376 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.973294 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:18.973376 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.973320 2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-188.ec2.internal" Sep 18 16:49:18.981169 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:18.981150 2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-188.ec2.internal" Sep 18 16:49:18.981252 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.981177 2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-188.ec2.internal\": node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:18.993427 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:18.993405 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.010862 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.010826 2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal"] Sep 18 16:49:19.010941 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.010909 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:19.011998 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.011982 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:19.012066 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.012015 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:19.012066 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.012026 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:19.014160 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.014145 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:19.014298 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.014281 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.014342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.014318 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:19.015026 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.015009 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:19.015026 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.015020 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:19.015148 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.015040 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:19.015148 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.015045 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:19.015148 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.015054 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:19.015148 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.015055 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:19.017432 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.017414 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.017505 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.017448 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 18 16:49:19.018318 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.018301 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientMemory" Sep 18 16:49:19.018398 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.018325 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasNoDiskPressure" Sep 18 16:49:19.018398 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.018335 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeHasSufficientPID" Sep 18 16:49:19.033322 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.033294 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-188.ec2.internal\" not found" node="ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.037491 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.037472 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-188.ec2.internal\" not found" node="ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.094062 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.094009 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.108806 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.108775 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99da832300d4ddecffd299754cc04b87-config\") pod \"kube-apiserver-proxy-ip-10-0-135-188.ec2.internal\" (UID: \"99da832300d4ddecffd299754cc04b87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.108905 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.108810 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33170ae1c2859d896bd08d20995c552c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" (UID: \"33170ae1c2859d896bd08d20995c552c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.108905 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.108833 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33170ae1c2859d896bd08d20995c552c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" (UID: \"33170ae1c2859d896bd08d20995c552c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.194455 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.194417 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.209723 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.209701 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99da832300d4ddecffd299754cc04b87-config\") pod \"kube-apiserver-proxy-ip-10-0-135-188.ec2.internal\" (UID: \"99da832300d4ddecffd299754cc04b87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.209798 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.209730 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33170ae1c2859d896bd08d20995c552c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" (UID: \"33170ae1c2859d896bd08d20995c552c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.209798 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.209748 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33170ae1c2859d896bd08d20995c552c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" (UID: \"33170ae1c2859d896bd08d20995c552c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.209877 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.209805 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99da832300d4ddecffd299754cc04b87-config\") pod \"kube-apiserver-proxy-ip-10-0-135-188.ec2.internal\" (UID: \"99da832300d4ddecffd299754cc04b87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.209877 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.209815 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33170ae1c2859d896bd08d20995c552c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" (UID: \"33170ae1c2859d896bd08d20995c552c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.209948 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.209872 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/33170ae1c2859d896bd08d20995c552c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" (UID: \"33170ae1c2859d896bd08d20995c552c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.295424 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.295359 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.337892 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.337855 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.342045 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.342024 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:19.395482 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.395441 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.496064 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.496024 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.596702 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.596661 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.647530 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.647502 2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 16:49:19.697764 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.697738 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.732207 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.732185 2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 18 16:49:19.732705 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.732304 2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 16:49:19.732705 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.732345 2586 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.135.188:57638->52.70.54.4:6443: use of closed network connection" Sep 18 16:49:19.732705 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.732359 2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 18 16:49:19.732705 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.732435 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: unexpected error when reading response body. Please retry. Original error: read tcp 10.0.135.188:57638->52.70.54.4:6443: use of closed network connection" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 18 16:49:19.797855 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.797815 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.821320 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.821273 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 16:44:18 +0000 UTC" deadline="2028-03-19 02:54:49.805867875 +0000 UTC" Sep 18 16:49:19.821320 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.821320 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13138h5m29.984552784s" Sep 18 16:49:19.898881 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.898805 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:19.980097 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:19.980062 2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 16:49:19.999398 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:19.999363 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.039057 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:20.039008 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99da832300d4ddecffd299754cc04b87.slice/crio-31d869c1f3a449098410ae25947034a049cf41bc695cd542ff048453ce0dd08b WatchSource:0}: Error finding container 31d869c1f3a449098410ae25947034a049cf41bc695cd542ff048453ce0dd08b: Status 404 returned error can't find the container with id 31d869c1f3a449098410ae25947034a049cf41bc695cd542ff048453ce0dd08b Sep 18 16:49:20.039680 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:20.039654 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33170ae1c2859d896bd08d20995c552c.slice/crio-6a404d7b712c541bbce931103a9daa8d81f20ffbf938c598043065823fd1b8b4 WatchSource:0}: Error finding container 6a404d7b712c541bbce931103a9daa8d81f20ffbf938c598043065823fd1b8b4: Status 404 returned error can't find the container with id 6a404d7b712c541bbce931103a9daa8d81f20ffbf938c598043065823fd1b8b4 Sep 18 16:49:20.044746 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:20.044723 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 16:49:20.100491 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.100447 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.201115 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.201036 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.302038 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.302002 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.402631 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.402592 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.503311 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.503275 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.603998 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.603937 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.627211 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:20.627025 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 16:49:20.704479 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.704394 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.805508 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.805470 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.906244 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:20.906206 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:20.915262 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:20.915196 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" event={"ID":"33170ae1c2859d896bd08d20995c552c","Type":"ContainerStarted","Data":"6a404d7b712c541bbce931103a9daa8d81f20ffbf938c598043065823fd1b8b4"} Sep 18 16:49:20.916364 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:20.916341 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" event={"ID":"99da832300d4ddecffd299754cc04b87","Type":"ContainerStarted","Data":"31d869c1f3a449098410ae25947034a049cf41bc695cd542ff048453ce0dd08b"} Sep 18 16:49:21.006868 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.006785 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.107292 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.107258 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.207848 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.207816 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.308705 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.308614 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.409722 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.409687 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.510325 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.510293 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.610972 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.610934 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.711641 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.711599 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.812706 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.812674 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:21.913222 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:21.913194 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:22.014357 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.014317 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:22.115351 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.115284 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:22.216103 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.215912 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:22.316891 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.316846 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-188.ec2.internal\" not found" Sep 18 16:49:22.365105 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.365084 2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 16:49:22.382439 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.382379 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:22.401221 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.401200 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 16:49:22.402134 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.402118 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:22.502121 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.502095 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 16:49:22.762400 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.762320 2586 apiserver.go:52] "Watching apiserver" Sep 18 16:49:22.770638 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.770611 2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 18 16:49:22.770962 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.770942 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-8hzb9","kube-system/global-pull-secret-syncer-9hg2q","kube-system/konnectivity-agent-tbjtj","kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7rsm4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal","openshift-multus/multus-285nx","openshift-multus/multus-additional-cni-plugins-8kdgf","openshift-multus/network-metrics-daemon-5jqgt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw","openshift-image-registry/node-ca-4x25g","openshift-network-diagnostics/network-check-target-kld7t","openshift-network-operator/iptables-alerter-hrpjk"] Sep 18 16:49:22.773509 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.773484 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.774322 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.774302 2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 18 16:49:22.777151 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.776123 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.777151 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.776733 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.777324 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.777214 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 18 16:49:22.777324 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.777264 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 18 16:49:22.777495 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.777478 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 18 16:49:22.777715 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.777699 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 18 16:49:22.777917 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.777886 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.778101 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.778084 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5qjbt\"" Sep 18 16:49:22.778513 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.778493 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.779330 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.779195 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 18 16:49:22.779330 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.779316 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.779452 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.779352 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-285nx" Sep 18 16:49:22.779452 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.779384 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-72crf\"" Sep 18 16:49:22.781660 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.781641 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.782649 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.782630 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 18 16:49:22.782757 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.782666 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9nqlx\"" Sep 18 16:49:22.782757 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.782711 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 18 16:49:22.782757 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.782674 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.782927 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.782671 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.783523 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.783506 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 18 16:49:22.783961 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.783947 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.784044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.783985 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8vfz2\"" Sep 18 16:49:22.784101 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.784047 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.784604 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.784583 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.786822 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.786804 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k4885\"" Sep 18 16:49:22.786959 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.786941 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.787049 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.786998 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 18 16:49:22.787049 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.787022 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 18 16:49:22.789118 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.789102 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:22.789573 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.789214 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:22.789573 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.789297 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s48bl\"" Sep 18 16:49:22.789573 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.789348 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 18 16:49:22.789573 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.789445 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 18 16:49:22.791439 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.791422 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.791518 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.791499 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:22.793884 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.793867 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.796175 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.796160 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:22.796250 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.796216 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:22.796335 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.796317 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.796446 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.796422 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.796589 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.796572 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 18 16:49:22.796661 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.796652 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rvczh\"" Sep 18 16:49:22.798665 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.798651 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-vvmcg"] Sep 18 16:49:22.798750 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.798732 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.801213 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.801196 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.801994 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.801972 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 18 16:49:22.802081 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.801997 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.802081 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.801973 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.802451 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.802426 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p4vr7\"" Sep 18 16:49:22.802546 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.802527 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 18 16:49:22.803437 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.803418 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 18 16:49:22.803437 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.803425 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jxt4w\"" Sep 18 16:49:22.803527 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.803448 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 18 16:49:22.807482 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.807466 2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 18 16:49:22.835402 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835375 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-node-log\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835411 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/be06cb5e-12f9-4bf7-b5ce-eb68f9864842-konnectivity-ca\") pod \"konnectivity-agent-tbjtj\" (UID: \"be06cb5e-12f9-4bf7-b5ce-eb68f9864842\") " pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835429 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-cnibin\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835445 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-cni-multus\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835467 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-run\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835483 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-lib-modules\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835498 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zdkc\" (UniqueName: \"kubernetes.io/projected/cde0ff74-2fda-4d8b-acec-53712d285b96-kube-api-access-6zdkc\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835531 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrffb\" (UniqueName: \"kubernetes.io/projected/39d524bb-bc47-49a4-a8d9-27217d803307-kube-api-access-zrffb\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835585 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-tuned\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835605 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e13e20ef-3426-456b-aaa7-05eff7e64740-iptables-alerter-script\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835632 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-var-lib-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835664 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2aace0d-8144-4803-b013-63eeaf617aca-dbus\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835702 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-k8s-cni-cncf-io\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835749 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-kubelet\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835781 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-modprobe-d\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835808 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-systemd-units\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.835896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835829 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-systemd\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835861 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovnkube-config\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835875 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2aace0d-8144-4803-b013-63eeaf617aca-kubelet-config\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835891 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-cni-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835907 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-netns\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835922 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysconfig\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835935 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysctl-d\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835948 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-slash\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835963 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4tgf\" (UniqueName: \"kubernetes.io/projected/871e64c9-5bf6-4560-aac6-6403d5f56df6-kube-api-access-l4tgf\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.835977 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysctl-conf\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836010 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-systemd\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836031 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64t8w\" (UniqueName: \"kubernetes.io/projected/dad9b91f-014d-4fdc-a1f3-838a7df5196d-kube-api-access-64t8w\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836045 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmq9p\" (UniqueName: \"kubernetes.io/projected/e13e20ef-3426-456b-aaa7-05eff7e64740-kube-api-access-mmq9p\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836063 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-log-socket\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836083 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chrsg\" (UniqueName: \"kubernetes.io/projected/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-kube-api-access-chrsg\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836099 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-etc-selinux\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.836351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836149 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-run-netns\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836167 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-sys-fs\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836181 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-multus-certs\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836202 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-conf-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836228 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cde0ff74-2fda-4d8b-acec-53712d285b96-tmp\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836247 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-cnibin\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836264 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwsrg\" (UniqueName: \"kubernetes.io/projected/581ad636-7b18-43bc-803c-8c163b820c0f-kube-api-access-vwsrg\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836282 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad9b91f-014d-4fdc-a1f3-838a7df5196d-tmp-dir\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836296 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovn-node-metrics-cert\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836310 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-socket-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836322 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-sys\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836336 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836373 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-cni-bin\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836401 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836423 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/be06cb5e-12f9-4bf7-b5ce-eb68f9864842-agent-certs\") pod \"konnectivity-agent-tbjtj\" (UID: \"be06cb5e-12f9-4bf7-b5ce-eb68f9864842\") " pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836441 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-hostroot\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.836826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836455 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-etc-kubernetes\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836468 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836485 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovnkube-script-lib\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836506 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-device-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836522 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836542 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fghvb\" (UniqueName: \"kubernetes.io/projected/776f65d4-9fed-4694-8139-fb8bb9a95c7b-kube-api-access-fghvb\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836577 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srd26\" (UniqueName: \"kubernetes.io/projected/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-kube-api-access-srd26\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836601 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-var-lib-kubelet\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836618 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836656 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-ovn\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836676 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836691 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836713 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/871e64c9-5bf6-4560-aac6-6403d5f56df6-serviceca\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836729 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-system-cni-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836742 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-cni-bin\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836755 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-daemon-config\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836768 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dad9b91f-014d-4fdc-a1f3-838a7df5196d-hosts-file\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836794 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-cni-netd\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836812 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-kubelet\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836826 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-etc-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836844 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-env-overrides\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836871 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/871e64c9-5bf6-4560-aac6-6403d5f56df6-host\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836893 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836977 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-cni-binary-copy\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.836999 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-socket-dir-parent\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837014 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-registration-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837031 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-kubernetes\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837048 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-host\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837070 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-system-cni-dir\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837087 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837106 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837131 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e13e20ef-3426-456b-aaa7-05eff7e64740-host-slash\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.837753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.837151 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.893440 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.893410 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5lvmd" Sep 18 16:49:22.904129 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.904108 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5lvmd" Sep 18 16:49:22.921737 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.921703 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" event={"ID":"33170ae1c2859d896bd08d20995c552c","Type":"ContainerStarted","Data":"ef7bccc0f908a31ef85523f75c69b5fa77bab7afd5c0f5d7418bdb120c44969b"} Sep 18 16:49:22.921910 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.921881 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:22.923068 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.923045 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" event={"ID":"99da832300d4ddecffd299754cc04b87","Type":"ContainerStarted","Data":"54105c897ff74e03887da2331e92d38edcf56cbb11d2396d552f394bbc63d729"} Sep 18 16:49:22.923222 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.923211 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:22.934283 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.934259 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 16:49:22.934386 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.934310 2586 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-188.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" Sep 18 16:49:22.935347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.935334 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 18 16:49:22.935393 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.935368 2586 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" Sep 18 16:49:22.937428 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937413 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4tgf\" (UniqueName: \"kubernetes.io/projected/871e64c9-5bf6-4560-aac6-6403d5f56df6-kube-api-access-l4tgf\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.937471 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937439 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysctl-conf\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.937471 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937454 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-systemd\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.937545 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937469 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64t8w\" (UniqueName: \"kubernetes.io/projected/dad9b91f-014d-4fdc-a1f3-838a7df5196d-kube-api-access-64t8w\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.937545 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937492 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmq9p\" (UniqueName: \"kubernetes.io/projected/e13e20ef-3426-456b-aaa7-05eff7e64740-kube-api-access-mmq9p\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.937545 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937514 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-log-socket\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.937545 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937535 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chrsg\" (UniqueName: \"kubernetes.io/projected/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-kube-api-access-chrsg\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.937722 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937546 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-systemd\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.937722 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937636 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-log-socket\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.937722 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937642 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysctl-conf\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.937722 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937687 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-etc-selinux\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.937722 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937718 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-run-netns\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937749 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-run-netns\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937775 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-sys-fs\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937819 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-multus-certs\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937831 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-etc-selinux\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937849 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-conf-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937868 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-sys-fs\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937874 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cde0ff74-2fda-4d8b-acec-53712d285b96-tmp\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937882 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-conf-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937886 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-multus-certs\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937912 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-cnibin\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.937936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937937 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwsrg\" (UniqueName: \"kubernetes.io/projected/581ad636-7b18-43bc-803c-8c163b820c0f-kube-api-access-vwsrg\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937957 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad9b91f-014d-4fdc-a1f3-838a7df5196d-tmp-dir\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937988 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-cnibin\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.937974 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovn-node-metrics-cert\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938041 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-socket-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938062 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-sys\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938087 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938166 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-sys\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938213 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-socket-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938253 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-cni-bin\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938280 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938286 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dad9b91f-014d-4fdc-a1f3-838a7df5196d-tmp-dir\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938308 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-cni-bin\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938281 2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938317 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/be06cb5e-12f9-4bf7-b5ce-eb68f9864842-agent-certs\") pod \"konnectivity-agent-tbjtj\" (UID: \"be06cb5e-12f9-4bf7-b5ce-eb68f9864842\") " pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.938342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938348 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-hostroot\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938365 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-etc-kubernetes\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938373 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938385 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938413 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovnkube-script-lib\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938423 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-etc-kubernetes\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938429 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-hostroot\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938434 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-device-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938467 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938470 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938494 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fghvb\" (UniqueName: \"kubernetes.io/projected/776f65d4-9fed-4694-8139-fb8bb9a95c7b-kube-api-access-fghvb\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938515 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-device-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938523 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srd26\" (UniqueName: \"kubernetes.io/projected/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-kube-api-access-srd26\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938575 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-var-lib-kubelet\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938603 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938629 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-ovn\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938665 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.939087 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938681 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938690 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938721 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/871e64c9-5bf6-4560-aac6-6403d5f56df6-serviceca\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938783 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-system-cni-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938812 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-cni-bin\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938839 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-daemon-config\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938864 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dad9b91f-014d-4fdc-a1f3-838a7df5196d-hosts-file\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938889 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-cni-netd\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938914 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-kubelet\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938939 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-etc-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938962 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-env-overrides\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.938985 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/871e64c9-5bf6-4560-aac6-6403d5f56df6-host\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939008 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939072 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-cni-binary-copy\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939097 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-socket-dir-parent\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939121 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-registration-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939143 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-kubernetes\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939163 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/871e64c9-5bf6-4560-aac6-6403d5f56df6-serviceca\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.940044 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939218 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-cni-netd\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939229 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dad9b91f-014d-4fdc-a1f3-838a7df5196d-hosts-file\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939266 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-kubelet\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939310 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-etc-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939356 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-var-lib-kubelet\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939409 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-daemon-config\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939468 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-system-cni-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939507 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-cni-bin\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939704 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-env-overrides\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939779 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-socket-dir-parent\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939839 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/39d524bb-bc47-49a4-a8d9-27217d803307-registration-dir\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939888 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-kubernetes\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.939957 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939997 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.940040 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:49:23.440001811 +0000 UTC m=+6.716126965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940056 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-ovn\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940066 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/871e64c9-5bf6-4560-aac6-6403d5f56df6-host\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.940894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940072 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-run-ovn-kubernetes\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940090 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovnkube-script-lib\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940112 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-host\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.940117 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.940179 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:49:23.440161933 +0000 UTC m=+6.716287088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.939167 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-host\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940223 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-system-cni-dir\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940249 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940287 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940287 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-system-cni-dir\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940318 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e13e20ef-3426-456b-aaa7-05eff7e64740-host-slash\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940343 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940365 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-node-log\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940388 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/be06cb5e-12f9-4bf7-b5ce-eb68f9864842-konnectivity-ca\") pod \"konnectivity-agent-tbjtj\" (UID: \"be06cb5e-12f9-4bf7-b5ce-eb68f9864842\") " pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940393 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e13e20ef-3426-456b-aaa7-05eff7e64740-host-slash\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940412 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940436 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-cnibin\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.941680 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940455 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-node-log\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940460 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-cni-multus\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940499 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-cni-multus\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940521 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-cnibin\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940549 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-run\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940600 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-lib-modules\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940615 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-run\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940624 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zdkc\" (UniqueName: \"kubernetes.io/projected/cde0ff74-2fda-4d8b-acec-53712d285b96-kube-api-access-6zdkc\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940716 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-lib-modules\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940724 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/581ad636-7b18-43bc-803c-8c163b820c0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940752 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-cni-binary-copy\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940800 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrffb\" (UniqueName: \"kubernetes.io/projected/39d524bb-bc47-49a4-a8d9-27217d803307-kube-api-access-zrffb\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940826 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-tuned\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940864 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e13e20ef-3426-456b-aaa7-05eff7e64740-iptables-alerter-script\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940888 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-var-lib-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940909 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2aace0d-8144-4803-b013-63eeaf617aca-dbus\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940933 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/be06cb5e-12f9-4bf7-b5ce-eb68f9864842-konnectivity-ca\") pod \"konnectivity-agent-tbjtj\" (UID: \"be06cb5e-12f9-4bf7-b5ce-eb68f9864842\") " pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940946 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-k8s-cni-cncf-io\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942347 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940969 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-kubelet\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.940994 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-modprobe-d\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941004 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-var-lib-openvswitch\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941035 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-systemd-units\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941059 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-systemd\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941103 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovnkube-config\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941131 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2aace0d-8144-4803-b013-63eeaf617aca-kubelet-config\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941156 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-cni-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941199 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-netns\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941199 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b2aace0d-8144-4803-b013-63eeaf617aca-dbus\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941239 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-k8s-cni-cncf-io\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941263 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-var-lib-kubelet\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941272 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysconfig\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941312 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysctl-d\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941313 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysconfig\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941360 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-multus-cni-dir\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941390 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-slash\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941406 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-modprobe-d\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.942858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941455 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-host-slash\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941465 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e13e20ef-3426-456b-aaa7-05eff7e64740-iptables-alerter-script\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941495 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-systemd-units\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941508 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b2aace0d-8144-4803-b013-63eeaf617aca-kubelet-config\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941515 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-host-run-netns\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941549 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-run-systemd\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941629 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-sysctl-d\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941648 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/581ad636-7b18-43bc-803c-8c163b820c0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.941798 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovnkube-config\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.942267 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-ovn-node-metrics-cert\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.942316 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cde0ff74-2fda-4d8b-acec-53712d285b96-tmp\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.943412 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.942332 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/be06cb5e-12f9-4bf7-b5ce-eb68f9864842-agent-certs\") pod \"konnectivity-agent-tbjtj\" (UID: \"be06cb5e-12f9-4bf7-b5ce-eb68f9864842\") " pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:22.943792 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.943446 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cde0ff74-2fda-4d8b-acec-53712d285b96-etc-tuned\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.949569 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.949197 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:22.949569 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.949223 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:22.949569 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.949237 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:22.949569 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:22.949335 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:49:23.449315053 +0000 UTC m=+6.725440226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:22.953110 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.953080 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64t8w\" (UniqueName: \"kubernetes.io/projected/dad9b91f-014d-4fdc-a1f3-838a7df5196d-kube-api-access-64t8w\") pod \"node-resolver-vvmcg\" (UID: \"dad9b91f-014d-4fdc-a1f3-838a7df5196d\") " pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:22.953236 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.953107 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmq9p\" (UniqueName: \"kubernetes.io/projected/e13e20ef-3426-456b-aaa7-05eff7e64740-kube-api-access-mmq9p\") pod \"iptables-alerter-hrpjk\" (UID: \"e13e20ef-3426-456b-aaa7-05eff7e64740\") " pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:22.953626 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.953601 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4tgf\" (UniqueName: \"kubernetes.io/projected/871e64c9-5bf6-4560-aac6-6403d5f56df6-kube-api-access-l4tgf\") pod \"node-ca-4x25g\" (UID: \"871e64c9-5bf6-4560-aac6-6403d5f56df6\") " pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:22.953626 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.953615 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fghvb\" (UniqueName: \"kubernetes.io/projected/776f65d4-9fed-4694-8139-fb8bb9a95c7b-kube-api-access-fghvb\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:22.954156 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.954115 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srd26\" (UniqueName: \"kubernetes.io/projected/1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3-kube-api-access-srd26\") pod \"multus-285nx\" (UID: \"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3\") " pod="openshift-multus/multus-285nx" Sep 18 16:49:22.954431 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.954408 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chrsg\" (UniqueName: \"kubernetes.io/projected/1297f04c-ff7b-4c98-a6ee-9fa6b887a561-kube-api-access-chrsg\") pod \"ovnkube-node-8hzb9\" (UID: \"1297f04c-ff7b-4c98-a6ee-9fa6b887a561\") " pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:22.954503 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.954487 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwsrg\" (UniqueName: \"kubernetes.io/projected/581ad636-7b18-43bc-803c-8c163b820c0f-kube-api-access-vwsrg\") pod \"multus-additional-cni-plugins-8kdgf\" (UID: \"581ad636-7b18-43bc-803c-8c163b820c0f\") " pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:22.954825 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.954808 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zdkc\" (UniqueName: \"kubernetes.io/projected/cde0ff74-2fda-4d8b-acec-53712d285b96-kube-api-access-6zdkc\") pod \"tuned-7rsm4\" (UID: \"cde0ff74-2fda-4d8b-acec-53712d285b96\") " pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:22.954871 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:22.954821 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrffb\" (UniqueName: \"kubernetes.io/projected/39d524bb-bc47-49a4-a8d9-27217d803307-kube-api-access-zrffb\") pod \"aws-ebs-csi-driver-node-fhxbw\" (UID: \"39d524bb-bc47-49a4-a8d9-27217d803307\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:23.002683 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.002611 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-188.ec2.internal" podStartSLOduration=1.002594137 podStartE2EDuration="1.002594137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:22.986235008 +0000 UTC m=+6.262360181" watchObservedRunningTime="2026-09-18 16:49:23.002594137 +0000 UTC m=+6.278719311" Sep 18 16:49:23.085003 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.084975 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:23.091515 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.091498 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" Sep 18 16:49:23.095534 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.095510 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1297f04c_ff7b_4c98_a6ee_9fa6b887a561.slice/crio-b32a806879d1dfa1b7e462c7480c2ee1752271b6dfe72360db4ea1008d558630 WatchSource:0}: Error finding container b32a806879d1dfa1b7e462c7480c2ee1752271b6dfe72360db4ea1008d558630: Status 404 returned error can't find the container with id b32a806879d1dfa1b7e462c7480c2ee1752271b6dfe72360db4ea1008d558630 Sep 18 16:49:23.096333 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.096316 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:23.100754 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.100727 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d524bb_bc47_49a4_a8d9_27217d803307.slice/crio-e9d31c4c5fc885ce7035e47c19e70d17489cd376096e406d4fb680dd714b3c23 WatchSource:0}: Error finding container e9d31c4c5fc885ce7035e47c19e70d17489cd376096e406d4fb680dd714b3c23: Status 404 returned error can't find the container with id e9d31c4c5fc885ce7035e47c19e70d17489cd376096e406d4fb680dd714b3c23 Sep 18 16:49:23.101327 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.101312 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-285nx" Sep 18 16:49:23.106636 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.106615 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe06cb5e_12f9_4bf7_b5ce_eb68f9864842.slice/crio-7179e275e92e7a8caa898cafbb06fbccf07ac8fe273321a1a72736f6547f00d8 WatchSource:0}: Error finding container 7179e275e92e7a8caa898cafbb06fbccf07ac8fe273321a1a72736f6547f00d8: Status 404 returned error can't find the container with id 7179e275e92e7a8caa898cafbb06fbccf07ac8fe273321a1a72736f6547f00d8 Sep 18 16:49:23.107293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.107277 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" Sep 18 16:49:23.110857 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.110837 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccfd74a_5cdf_48ff_af9c_f1f56c5e55f3.slice/crio-f9c5e75d8066c022a3ea2893634711b04390b878cf61037cf6de54b0939041fa WatchSource:0}: Error finding container f9c5e75d8066c022a3ea2893634711b04390b878cf61037cf6de54b0939041fa: Status 404 returned error can't find the container with id f9c5e75d8066c022a3ea2893634711b04390b878cf61037cf6de54b0939041fa Sep 18 16:49:23.113414 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.113119 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" Sep 18 16:49:23.117826 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.117802 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde0ff74_2fda_4d8b_acec_53712d285b96.slice/crio-dd13c287ac298f6f57ee231c255dbd838441b491a63301b2a8b9f261b6e44dfb WatchSource:0}: Error finding container dd13c287ac298f6f57ee231c255dbd838441b491a63301b2a8b9f261b6e44dfb: Status 404 returned error can't find the container with id dd13c287ac298f6f57ee231c255dbd838441b491a63301b2a8b9f261b6e44dfb Sep 18 16:49:23.118357 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.118338 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4x25g" Sep 18 16:49:23.123999 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.123982 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hrpjk" Sep 18 16:49:23.127427 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.127413 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vvmcg" Sep 18 16:49:23.178500 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.178455 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581ad636_7b18_43bc_803c_8c163b820c0f.slice/crio-514422953ac67aecb14d9b7ae8e82cfddff2dd7ed2b4144d94178e980856b578 WatchSource:0}: Error finding container 514422953ac67aecb14d9b7ae8e82cfddff2dd7ed2b4144d94178e980856b578: Status 404 returned error can't find the container with id 514422953ac67aecb14d9b7ae8e82cfddff2dd7ed2b4144d94178e980856b578 Sep 18 16:49:23.178743 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.178724 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad9b91f_014d_4fdc_a1f3_838a7df5196d.slice/crio-fec04b398da1d7d2aed8d506247f8aeebbc63ed4bccf24962609371d60967dd6 WatchSource:0}: Error finding container fec04b398da1d7d2aed8d506247f8aeebbc63ed4bccf24962609371d60967dd6: Status 404 returned error can't find the container with id fec04b398da1d7d2aed8d506247f8aeebbc63ed4bccf24962609371d60967dd6 Sep 18 16:49:23.179134 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.178997 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13e20ef_3426_456b_aaa7_05eff7e64740.slice/crio-1664b28aac9049fedee1d8a64755ddc443a3fa31b0edff6c03d8d685c5f4ae2d WatchSource:0}: Error finding container 1664b28aac9049fedee1d8a64755ddc443a3fa31b0edff6c03d8d685c5f4ae2d: Status 404 returned error can't find the container with id 1664b28aac9049fedee1d8a64755ddc443a3fa31b0edff6c03d8d685c5f4ae2d Sep 18 16:49:23.179208 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:49:23.179193 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871e64c9_5bf6_4560_aac6_6403d5f56df6.slice/crio-0a73d04a13e603d3b22bf4e2450c859488604716bb4d54a1885e3fb7bd514d8b WatchSource:0}: Error finding container 0a73d04a13e603d3b22bf4e2450c859488604716bb4d54a1885e3fb7bd514d8b: Status 404 returned error can't find the container with id 0a73d04a13e603d3b22bf4e2450c859488604716bb4d54a1885e3fb7bd514d8b Sep 18 16:49:23.444822 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.444730 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:23.444822 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.444770 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:23.445014 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.444877 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:23.445014 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.444888 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:23.445014 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.444944 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:49:24.444929203 +0000 UTC m=+7.721054353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:23.445014 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.444963 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:49:24.444952583 +0000 UTC m=+7.721077733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:23.545661 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.545624 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:23.545833 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.545768 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:23.545833 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.545785 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:23.545833 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.545795 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:23.545993 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.545848 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:49:24.545834679 +0000 UTC m=+7.821959849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:23.907653 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.907604 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 16:44:22 +0000 UTC" deadline="2028-06-01 18:08:36.081127476 +0000 UTC" Sep 18 16:49:23.907653 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.907648 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14929h19m12.173484031s" Sep 18 16:49:23.911065 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.911039 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:23.911208 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:23.911166 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:23.929654 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.929609 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vvmcg" event={"ID":"dad9b91f-014d-4fdc-a1f3-838a7df5196d","Type":"ContainerStarted","Data":"fec04b398da1d7d2aed8d506247f8aeebbc63ed4bccf24962609371d60967dd6"} Sep 18 16:49:23.932977 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.932900 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerStarted","Data":"514422953ac67aecb14d9b7ae8e82cfddff2dd7ed2b4144d94178e980856b578"} Sep 18 16:49:23.934867 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.934838 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" event={"ID":"cde0ff74-2fda-4d8b-acec-53712d285b96","Type":"ContainerStarted","Data":"dd13c287ac298f6f57ee231c255dbd838441b491a63301b2a8b9f261b6e44dfb"} Sep 18 16:49:23.937512 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.937482 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tbjtj" event={"ID":"be06cb5e-12f9-4bf7-b5ce-eb68f9864842","Type":"ContainerStarted","Data":"7179e275e92e7a8caa898cafbb06fbccf07ac8fe273321a1a72736f6547f00d8"} Sep 18 16:49:23.944667 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.944508 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" event={"ID":"39d524bb-bc47-49a4-a8d9-27217d803307","Type":"ContainerStarted","Data":"e9d31c4c5fc885ce7035e47c19e70d17489cd376096e406d4fb680dd714b3c23"} Sep 18 16:49:23.950961 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.950818 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"b32a806879d1dfa1b7e462c7480c2ee1752271b6dfe72360db4ea1008d558630"} Sep 18 16:49:23.971101 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.971057 2586 generic.go:356] "Generic (PLEG): container finished" podID="33170ae1c2859d896bd08d20995c552c" containerID="ef7bccc0f908a31ef85523f75c69b5fa77bab7afd5c0f5d7418bdb120c44969b" exitCode=0 Sep 18 16:49:23.971270 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.971213 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" event={"ID":"33170ae1c2859d896bd08d20995c552c","Type":"ContainerDied","Data":"ef7bccc0f908a31ef85523f75c69b5fa77bab7afd5c0f5d7418bdb120c44969b"} Sep 18 16:49:23.982514 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.982048 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hrpjk" event={"ID":"e13e20ef-3426-456b-aaa7-05eff7e64740","Type":"ContainerStarted","Data":"1664b28aac9049fedee1d8a64755ddc443a3fa31b0edff6c03d8d685c5f4ae2d"} Sep 18 16:49:23.993006 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:23.992899 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4x25g" event={"ID":"871e64c9-5bf6-4560-aac6-6403d5f56df6","Type":"ContainerStarted","Data":"0a73d04a13e603d3b22bf4e2450c859488604716bb4d54a1885e3fb7bd514d8b"} Sep 18 16:49:24.009160 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.009086 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerStarted","Data":"f9c5e75d8066c022a3ea2893634711b04390b878cf61037cf6de54b0939041fa"} Sep 18 16:49:24.453028 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.452934 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:24.453028 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.452996 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:24.453271 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.453146 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:24.453271 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.453207 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:49:26.453189805 +0000 UTC m=+9.729314962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:24.453626 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.453606 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:24.453770 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.453658 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:49:26.453641943 +0000 UTC m=+9.729767093 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:24.553940 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.553903 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:24.554131 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.554117 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:24.554199 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.554138 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:24.554199 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.554153 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:24.554288 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.554227 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:49:26.554191529 +0000 UTC m=+9.830316686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:24.908913 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.908791 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 16:44:22 +0000 UTC" deadline="2028-03-20 23:09:36.184294819 +0000 UTC" Sep 18 16:49:24.908913 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.908832 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13182h20m11.275467611s" Sep 18 16:49:24.921577 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.916697 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:24.921577 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.916877 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:24.921577 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:24.917384 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:24.921577 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:24.917537 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:25.036476 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:25.035746 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" event={"ID":"33170ae1c2859d896bd08d20995c552c","Type":"ContainerStarted","Data":"d2c9894ace7b010c782897d64d9e99c17b9ffb697194402fd3b793a05fca4634"} Sep 18 16:49:25.050132 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:25.050066 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-188.ec2.internal" podStartSLOduration=3.050044225 podStartE2EDuration="3.050044225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:25.049629591 +0000 UTC m=+8.325754781" watchObservedRunningTime="2026-09-18 16:49:25.050044225 +0000 UTC m=+8.326169401" Sep 18 16:49:25.909964 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:25.909934 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:25.910437 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:25.910060 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:26.470994 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:26.470951 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:26.471183 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:26.471013 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:26.471183 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.471087 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:26.471183 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.471157 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:49:30.471140719 +0000 UTC m=+13.747265874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:26.471183 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.471161 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:26.471330 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.471213 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:49:30.471197562 +0000 UTC m=+13.747322727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:26.572038 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:26.571989 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:26.572238 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.572217 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:26.572382 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.572245 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:26.572382 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.572260 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:26.572382 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.572320 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:49:30.572304034 +0000 UTC m=+13.848429190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:26.912121 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:26.911649 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:26.912121 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.911767 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:26.912881 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:26.912703 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:26.912881 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:26.912824 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:27.910456 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:27.910421 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:27.910663 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:27.910572 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:28.911407 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:28.911369 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:28.911841 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:28.911507 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:28.911841 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:28.911574 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:28.911841 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:28.911691 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:29.910794 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:29.910678 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:29.910794 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:29.910784 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:30.505903 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:30.505669 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:30.505903 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:30.505729 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:30.505903 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.505832 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:30.505903 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.505847 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:30.505903 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.505897 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:49:38.505878867 +0000 UTC m=+21.782004032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:30.505903 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.505913 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:49:38.505905631 +0000 UTC m=+21.782030782 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:30.606734 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:30.606692 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:30.606935 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.606909 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:30.607028 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.606945 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:30.607028 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.606962 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:30.607128 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.607030 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:49:38.607011408 +0000 UTC m=+21.883136563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:30.910404 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:30.910366 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:30.910595 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:30.910366 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:30.910595 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.910498 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:30.910736 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:30.910599 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:31.910212 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:31.910166 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:31.910703 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:31.910313 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:32.910484 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:32.910443 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:32.910484 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:32.910475 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:32.911012 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:32.910602 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:32.911012 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:32.910689 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:33.910968 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:33.910928 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:33.911375 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:33.911043 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:34.910248 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:34.910212 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:34.910401 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:34.910354 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:34.910811 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:34.910794 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:34.910915 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:34.910895 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:35.910382 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:35.910341 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:35.910849 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:35.910474 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:36.910824 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:36.910788 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:36.910824 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:36.910808 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:36.911357 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:36.910971 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:36.911357 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:36.911090 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:37.910735 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:37.910690 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:37.910930 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:37.910863 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:38.570274 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:38.570238 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:38.570423 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:38.570309 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:38.570423 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.570393 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:38.570524 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.570427 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:38.570524 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.570457 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:49:54.57043571 +0000 UTC m=+37.846560864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:38.570524 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.570493 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:49:54.570474863 +0000 UTC m=+37.846600027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:38.671326 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:38.671292 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:38.671519 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.671428 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:38.671519 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.671445 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:38.671519 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.671453 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:38.671519 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.671511 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:49:54.671491517 +0000 UTC m=+37.947616688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:38.911446 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:38.910947 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:38.911446 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.911077 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:38.911446 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:38.911137 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:38.911446 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:38.911242 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:39.910783 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:39.910733 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:39.910982 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:39.910876 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:40.910674 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:40.910645 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:40.911079 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:40.910645 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:40.911079 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:40.910761 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:40.911079 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:40.910868 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:41.078048 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.077695 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vvmcg" event={"ID":"dad9b91f-014d-4fdc-a1f3-838a7df5196d","Type":"ContainerStarted","Data":"0f929aba051a1fc20af12cccc324d96b7962564d4625c40d56beec0c9085540f"} Sep 18 16:49:41.089076 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.089019 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" event={"ID":"cde0ff74-2fda-4d8b-acec-53712d285b96","Type":"ContainerStarted","Data":"18042d4ea451cb082980ef4cb0f46f45f8a98c05d004b6ecf445236819e7fa81"} Sep 18 16:49:41.092990 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.092928 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tbjtj" event={"ID":"be06cb5e-12f9-4bf7-b5ce-eb68f9864842","Type":"ContainerStarted","Data":"952629809adc9e9137b10cdfc8bfadf529c1a86ad251f4a098e9e803f862d59c"} Sep 18 16:49:41.096125 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.095749 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vvmcg" podStartSLOduration=1.426985143 podStartE2EDuration="19.095735665s" totalImagesPullingTime="17.668750522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:41.095041837 +0000 UTC m=+24.371167009" watchObservedRunningTime="2026-09-18 16:49:41.095735665 +0000 UTC m=+24.371860837" Sep 18 16:49:41.115723 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.115529 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7rsm4" podStartSLOduration=4.389903106 podStartE2EDuration="22.115505945s" totalImagesPullingTime="17.725602839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:41.115170729 +0000 UTC m=+24.391296120" watchObservedRunningTime="2026-09-18 16:49:41.115505945 +0000 UTC m=+24.391631124" Sep 18 16:49:41.133135 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.133078 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tbjtj" podStartSLOduration=11.83149829 podStartE2EDuration="22.133053381s" totalImagesPullingTime="10.301555091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:41.132328124 +0000 UTC m=+24.408453298" watchObservedRunningTime="2026-09-18 16:49:41.133053381 +0000 UTC m=+24.409178555" Sep 18 16:49:41.910697 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:41.910661 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:41.911460 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:41.910795 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:42.098645 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.098610 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4x25g" event={"ID":"871e64c9-5bf6-4560-aac6-6403d5f56df6","Type":"ContainerStarted","Data":"b0e3f3072519e01e651d1c8c4ce2d429e4b6ce91601fa2c78a287a5e5d03a0f0"} Sep 18 16:49:42.101082 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.101048 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerStarted","Data":"ac619034c78bcca7091c3424f793e9de568e8024dc8e2b352e285fcd578d94c2"} Sep 18 16:49:42.102542 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.102517 2586 generic.go:356] "Generic (PLEG): container finished" podID="581ad636-7b18-43bc-803c-8c163b820c0f" containerID="99f7b2f684edf5e0f0c70be94c72143d4a3ccd9afff027677e65e52f047b28ef" exitCode=0 Sep 18 16:49:42.102688 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.102595 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerDied","Data":"99f7b2f684edf5e0f0c70be94c72143d4a3ccd9afff027677e65e52f047b28ef"} Sep 18 16:49:42.108861 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.108824 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" event={"ID":"39d524bb-bc47-49a4-a8d9-27217d803307","Type":"ContainerStarted","Data":"e341f527a15e2ebf0ba628c4495c35cab0aea4c94352dc7e5a7049dd496b5102"} Sep 18 16:49:42.114440 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.114409 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"e242f692719fec3ef428c20709869ee7acc380f60ca6a7b9664846e84ee14d73"} Sep 18 16:49:42.114712 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.114501 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"0197e8867c93065d61d599b3a64dd89203bef9f852402be308058a6a21a5a8ea"} Sep 18 16:49:42.114712 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.114590 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"34aa2b37ea7c4898130038d270704550676323d3aed85147c7b9995c6f843835"} Sep 18 16:49:42.114712 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.114641 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"6b8cf58223aaa4df51cdc92fa4ef9b94c81b51b61d76a68ccbdb734bbbe543c0"} Sep 18 16:49:42.114712 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.114690 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"4511a1863d291183d63e79c2d4c17602d80e0ada3b2cc0d5f638c05672512712"} Sep 18 16:49:42.114883 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.114740 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"0f8c3947263a96865abcc9c30e81745f3b3725c6553ed5c0287aef7a40ad7e65"} Sep 18 16:49:42.122643 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.122597 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4x25g" podStartSLOduration=5.453860922 podStartE2EDuration="23.122580333s" totalImagesPullingTime="17.668719411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:42.122112456 +0000 UTC m=+25.398237629" watchObservedRunningTime="2026-09-18 16:49:42.122580333 +0000 UTC m=+25.398705499" Sep 18 16:49:42.141512 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.141469 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-285nx" podStartSLOduration=5.32614282 podStartE2EDuration="23.141452301s" totalImagesPullingTime="17.815309481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:42.141081566 +0000 UTC m=+25.417206760" watchObservedRunningTime="2026-09-18 16:49:42.141452301 +0000 UTC m=+25.417577509" Sep 18 16:49:42.163917 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.163843 2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 18 16:49:42.886182 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.885998 2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T16:49:42.163864045Z","UUID":"3af0908f-40f2-4c0c-b7e9-98c052c1824c","Handler":null,"Name":"","Endpoint":""} Sep 18 16:49:42.887764 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.887731 2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 18 16:49:42.887874 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.887782 2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 18 16:49:42.910603 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.910547 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:42.910793 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:42.910712 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:42.910793 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:42.910578 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:42.911253 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:42.910811 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:43.120079 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:43.120035 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" event={"ID":"39d524bb-bc47-49a4-a8d9-27217d803307","Type":"ContainerStarted","Data":"4b3d063c2cf596f41504ef09e5f6d2a5c5472cfadea54554d71f984449a814c1"} Sep 18 16:49:43.120265 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:43.120101 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" event={"ID":"39d524bb-bc47-49a4-a8d9-27217d803307","Type":"ContainerStarted","Data":"a4391861d0417aa55fe24b77558e9a2274820a2bd290bfab240ce0c8cac0aa70"} Sep 18 16:49:43.121826 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:43.121788 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hrpjk" event={"ID":"e13e20ef-3426-456b-aaa7-05eff7e64740","Type":"ContainerStarted","Data":"9f91c7be31df112da6c703baa538167ac70cc6b03de77a9d0134e94af8cff5e6"} Sep 18 16:49:43.138966 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:43.138866 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fhxbw" podStartSLOduration=5.5995266820000005 podStartE2EDuration="25.13885044s" totalImagesPullingTime="19.539323758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:43.13868842 +0000 UTC m=+26.414813616" watchObservedRunningTime="2026-09-18 16:49:43.13885044 +0000 UTC m=+26.414975613" Sep 18 16:49:43.157546 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:43.157482 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hrpjk" podStartSLOduration=6.488872095 podStartE2EDuration="24.157458665s" totalImagesPullingTime="17.66858657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:43.157245406 +0000 UTC m=+26.433370592" watchObservedRunningTime="2026-09-18 16:49:43.157458665 +0000 UTC m=+26.433583840" Sep 18 16:49:43.910744 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:43.910707 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:43.910915 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:43.910819 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:44.128227 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:44.128186 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"24ce0e016578e0e7f3318a0931c453b07a0d982d81f8f4baf6a49e14fddcfa3f"} Sep 18 16:49:44.899625 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:44.899578 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:44.900333 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:44.900310 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:44.910691 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:44.910661 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:44.910879 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:44.910660 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:44.910879 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:44.910799 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:44.910879 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:44.910856 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:45.131150 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:45.130942 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:45.131596 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:45.131576 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tbjtj" Sep 18 16:49:45.910940 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:45.910898 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:45.911375 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:45.911046 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:46.910532 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:46.910345 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:46.910785 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:46.910437 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:46.910785 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:46.910632 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:46.910785 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:46.910717 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:47.140420 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.140385 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"59dea95c912039f6d64fdb5cec75eea0ddcaef7087cc14c7fd839ae6eb14e8d3"} Sep 18 16:49:47.141228 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.140677 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:47.141228 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.140738 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:47.141228 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.140806 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:47.143003 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.142981 2586 generic.go:356] "Generic (PLEG): container finished" podID="581ad636-7b18-43bc-803c-8c163b820c0f" containerID="5874265c9bf81b9362db5db384305baf6b7d3578238c4cb2d836942a80361f25" exitCode=0 Sep 18 16:49:47.143127 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.143064 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerDied","Data":"5874265c9bf81b9362db5db384305baf6b7d3578238c4cb2d836942a80361f25"} Sep 18 16:49:47.158303 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.158275 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:47.158439 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.158377 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:49:47.169764 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.169712 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podStartSLOduration=11.396734357 podStartE2EDuration="29.169696726s" totalImagesPullingTime="17.772962369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:47.169229299 +0000 UTC m=+30.445354527" watchObservedRunningTime="2026-09-18 16:49:47.169696726 +0000 UTC m=+30.445821899" Sep 18 16:49:47.910813 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:47.910780 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:47.911029 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:47.910896 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:48.911210 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:48.911165 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:48.911210 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:48.911215 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:48.911859 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:48.911304 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:48.911859 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:48.911413 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:49.152485 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:49.152451 2586 generic.go:356] "Generic (PLEG): container finished" podID="581ad636-7b18-43bc-803c-8c163b820c0f" containerID="3549f2c2855286577ea518e87fe6d396a8b40b1dd20d54214d09a11dda8a0053" exitCode=0 Sep 18 16:49:49.152688 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:49.152536 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerDied","Data":"3549f2c2855286577ea518e87fe6d396a8b40b1dd20d54214d09a11dda8a0053"} Sep 18 16:49:49.910549 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:49.910512 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:49.910738 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:49.910635 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:50.157028 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:50.156994 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerStarted","Data":"09a6b8d83e0925f2591344e57590f8c25b1ea9c112662e92c6ff4d1bc667a56f"} Sep 18 16:49:50.910714 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:50.910681 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:50.910921 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:50.910682 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:50.910921 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:50.910793 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:50.910921 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:50.910872 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:51.161730 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:51.161638 2586 generic.go:356] "Generic (PLEG): container finished" podID="581ad636-7b18-43bc-803c-8c163b820c0f" containerID="09a6b8d83e0925f2591344e57590f8c25b1ea9c112662e92c6ff4d1bc667a56f" exitCode=0 Sep 18 16:49:51.161730 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:51.161702 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerDied","Data":"09a6b8d83e0925f2591344e57590f8c25b1ea9c112662e92c6ff4d1bc667a56f"} Sep 18 16:49:51.910309 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:51.910272 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:51.910493 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:51.910394 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:52.910813 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:52.910049 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:52.910813 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:52.910228 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:52.910813 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:52.910689 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:52.910813 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:52.910775 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:53.910357 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:53.910315 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:53.910596 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:53.910461 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:54.591618 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:54.591580 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:54.591618 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:54.591623 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:54.592331 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.591752 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:54.592331 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.591752 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:54.592331 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.591813 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:50:26.59179524 +0000 UTC m=+69.867920392 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:49:54.592331 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.591829 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:50:26.591820924 +0000 UTC m=+69.867946075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:49:54.692123 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:54.692070 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:54.692298 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.692250 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:49:54.692298 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.692274 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:49:54.692298 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.692284 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:54.692433 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.692347 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:50:26.692327793 +0000 UTC m=+69.968452960 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:49:54.910907 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:54.910814 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:54.911079 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.910980 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:54.911183 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:54.910814 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:54.911310 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:54.911269 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:55.910735 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:55.910700 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:55.911147 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:55.910808 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:56.178783 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:56.178701 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerStarted","Data":"19d260825a9902aa1951a6f5a999102ad750a9c4a1117f46019050de765fa7f0"} Sep 18 16:49:56.911140 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:56.911102 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:56.911595 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:56.911243 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:56.911595 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:56.911287 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:56.911595 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:56.911383 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:57.183693 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:57.183596 2586 generic.go:356] "Generic (PLEG): container finished" podID="581ad636-7b18-43bc-803c-8c163b820c0f" containerID="19d260825a9902aa1951a6f5a999102ad750a9c4a1117f46019050de765fa7f0" exitCode=0 Sep 18 16:49:57.183693 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:57.183670 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerDied","Data":"19d260825a9902aa1951a6f5a999102ad750a9c4a1117f46019050de765fa7f0"} Sep 18 16:49:57.910970 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:57.910932 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:57.911143 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:57.911040 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:49:58.189271 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:58.189188 2586 generic.go:356] "Generic (PLEG): container finished" podID="581ad636-7b18-43bc-803c-8c163b820c0f" containerID="70aef443130bca317644998ee063aabcc224ae2d363c582b2cc6deb68e083c26" exitCode=0 Sep 18 16:49:58.189271 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:58.189255 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerDied","Data":"70aef443130bca317644998ee063aabcc224ae2d363c582b2cc6deb68e083c26"} Sep 18 16:49:58.911185 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:58.911147 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:49:58.911383 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:58.911245 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:49:58.914186 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:58.911803 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:49:58.914186 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:58.911982 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:49:59.195609 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:59.195507 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" event={"ID":"581ad636-7b18-43bc-803c-8c163b820c0f","Type":"ContainerStarted","Data":"520301f1f9d82bfb51bbb5795df1ac65d386d30bec60fdaec9cae93691d50435"} Sep 18 16:49:59.223025 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:59.222976 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8kdgf" podStartSLOduration=11.214382586 podStartE2EDuration="40.222957456s" totalImagesPullingTime="29.00857487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:49:59.220515316 +0000 UTC m=+42.496640503" watchObservedRunningTime="2026-09-18 16:49:59.222957456 +0000 UTC m=+42.499082631" Sep 18 16:49:59.910135 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:49:59.910102 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:49:59.910309 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:49:59.910209 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:00.910821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:00.910780 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:00.911257 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:00.910780 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:00.911257 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:00.910930 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:00.911257 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:00.910980 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:01.910385 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:01.910335 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:01.910538 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:01.910459 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:02.910804 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:02.910758 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:02.911172 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:02.910909 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:02.911172 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:02.910955 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:02.911172 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:02.911040 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:03.910754 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:03.910708 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:03.910941 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:03.910843 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:04.910272 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:04.910238 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:04.910422 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:04.910244 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:04.910422 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:04.910343 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:04.910521 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:04.910422 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:05.910981 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:05.910942 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:05.911363 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:05.911064 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:06.910141 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:06.910095 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:06.910327 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:06.910095 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:06.910327 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:06.910241 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:06.910327 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:06.910301 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:07.910273 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:07.910233 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:07.910698 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:07.910342 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:08.911788 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:08.911744 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:08.912192 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:08.911839 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:08.912192 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:08.911938 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:08.912192 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:08.912087 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:09.910293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:09.910252 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:09.910480 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:09.910357 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:10.910758 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:10.910722 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:10.911137 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:10.910722 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:10.911137 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:10.910844 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:10.911137 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:10.910919 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:11.910786 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:11.910740 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:11.911172 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:11.910867 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:12.910858 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:12.910822 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:12.911214 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:12.910837 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:12.911214 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:12.910935 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:12.911214 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:12.910993 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:13.910834 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:13.910792 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:13.911028 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:13.910906 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:14.913433 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:14.913400 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:14.913433 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:14.913431 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:14.913850 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:14.913520 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:14.913850 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:14.913632 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:15.910967 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:15.910923 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:15.911269 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:15.911069 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:16.910603 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:16.910541 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:16.911078 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:16.910541 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:16.911078 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:16.910699 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:16.911078 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:16.910747 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:17.910684 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:17.910642 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:17.911058 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:17.910758 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:18.911633 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:18.911417 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:18.912113 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:18.911485 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:18.912113 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:18.911737 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:18.912113 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:18.911772 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:19.173743 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:19.173649 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:50:19.910637 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:19.910597 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:19.910849 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:19.910710 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:20.910726 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:20.910689 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:20.911135 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:20.910695 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:20.911135 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:20.910806 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:20.911135 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:20.910870 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:21.910187 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:21.910145 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:21.910395 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:21.910279 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:22.910568 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:22.910518 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:22.910980 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:22.910652 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:22.910980 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:22.910706 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:22.910980 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:22.910804 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:23.910277 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:23.910245 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:23.910463 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:23.910349 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:24.910950 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:24.910912 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:24.911437 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:24.911036 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:24.911437 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:24.911089 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:24.911437 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:24.911172 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:25.910911 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:25.910875 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:25.911103 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:25.910982 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:26.597353 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:26.597319 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:26.597507 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:26.597357 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:26.597507 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.597454 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:50:26.597507 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.597463 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:50:26.597639 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.597511 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:51:30.597494155 +0000 UTC m=+133.873619327 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:50:26.597639 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.597526 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:51:30.597519042 +0000 UTC m=+133.873644193 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:50:26.697819 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:26.697779 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:26.697986 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.697919 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:50:26.697986 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.697934 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:50:26.697986 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.697945 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:50:26.698087 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.697997 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:51:30.697984785 +0000 UTC m=+133.974109936 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:50:26.910479 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:26.910394 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:26.910648 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:26.910399 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:26.910648 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.910502 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:26.910648 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:26.910585 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:27.267669 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:27.267592 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/0.log" Sep 18 16:50:27.267669 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:27.267632 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" containerID="ac619034c78bcca7091c3424f793e9de568e8024dc8e2b352e285fcd578d94c2" exitCode=1 Sep 18 16:50:27.267669 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:27.267666 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerDied","Data":"ac619034c78bcca7091c3424f793e9de568e8024dc8e2b352e285fcd578d94c2"} Sep 18 16:50:27.268245 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:27.268013 2586 scope.go:117] "RemoveContainer" containerID="ac619034c78bcca7091c3424f793e9de568e8024dc8e2b352e285fcd578d94c2" Sep 18 16:50:27.910521 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:27.910477 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:27.910737 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:27.910610 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:28.272452 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:28.272364 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/0.log" Sep 18 16:50:28.272452 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:28.272421 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerStarted","Data":"4cdce6e4f9cb5810455b90031ce756896d39c2751668f6efa02389004ec4190f"} Sep 18 16:50:28.911626 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:28.911443 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:28.911626 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:28.911537 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:28.911626 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:28.911608 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:28.911924 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:28.911685 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:29.910315 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:29.910271 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:29.910759 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:29.910380 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:30.910910 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:30.910867 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:30.911330 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:30.910979 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:30.911330 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:30.911008 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:30.911330 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:30.911054 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:31.910719 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:31.910667 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:31.910989 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:31.910791 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:32.910368 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:32.910334 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:32.910549 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:32.910460 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:32.910625 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:32.910542 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:32.910625 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:32.910459 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:33.910020 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:33.909976 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:33.910405 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:33.910084 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:34.910472 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:34.910434 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:34.910897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:34.910442 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:34.910897 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:34.910586 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:34.910897 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:34.910658 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:35.910831 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:35.910789 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:35.911256 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:35.910904 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:36.910938 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:36.910902 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:36.911298 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:36.910910 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:36.911298 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:36.911032 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:36.911298 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:36.911104 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:37.910180 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:37.910143 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:37.910378 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:37.910251 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:38.913360 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:38.913318 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:38.914052 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:38.914014 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:38.914538 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:38.914424 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:38.914965 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:38.914941 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:39.911066 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:39.911025 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:39.911248 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:39.911139 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:40.910546 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:40.910502 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:40.911000 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:40.910510 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:40.911000 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:40.910643 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:40.911000 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:40.910712 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:41.910427 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:41.910393 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:41.910637 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:41.910501 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:42.910084 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:42.910038 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:42.910284 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:42.910042 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:42.910284 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:42.910159 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:42.910284 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:42.910243 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:43.400158 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:43.400134 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vvmcg_dad9b91f-014d-4fdc-a1f3-838a7df5196d/dns-node-resolver/0.log" Sep 18 16:50:43.911035 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:43.910996 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:43.911220 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:43.911101 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:44.001198 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:44.001172 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4x25g_871e64c9-5bf6-4560-aac6-6403d5f56df6/node-ca/0.log" Sep 18 16:50:44.910814 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:44.910774 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:44.911224 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:44.910777 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:44.911224 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:44.910893 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:44.911224 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:44.910949 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:45.910948 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:45.910908 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:45.911378 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:45.911020 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:46.910477 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:46.910438 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:46.910477 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:46.910465 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:46.910709 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:46.910572 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:46.910709 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:46.910695 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:47.910181 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:47.910130 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:47.910635 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:47.910248 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:48.911345 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:48.911297 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:48.911834 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:48.911414 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:48.911834 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:48.911415 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:48.911834 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:48.911531 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:49.163792 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:49.163705 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:50:49.910165 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:49.910113 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:49.910357 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:49.910239 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:50.910566 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:50.910528 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:50.910936 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:50.910592 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:50.910936 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:50.910672 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:50.910936 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:50.910764 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:51.910330 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:51.910289 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:51.910585 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:51.910537 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:52.910772 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:52.910729 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:52.911249 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:52.910729 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:52.911249 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:52.910842 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:52.911249 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:52.910960 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:53.911039 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:53.910999 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:53.911430 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:53.911199 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:54.910367 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:54.910336 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:54.910622 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:54.910342 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:54.910622 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:54.910481 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:54.910744 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:54.910623 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:55.910039 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:55.909999 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:55.910429 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:55.910109 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:56.910466 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:56.910424 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:56.910881 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:56.910424 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:56.910881 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:56.910544 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:56.910881 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:56.910616 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:57.910116 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:57.910062 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:57.910323 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:57.910183 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:50:58.913128 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:58.913093 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:50:58.913530 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:58.913137 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:50:58.913530 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:58.913200 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:50:58.913530 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:58.913238 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:50:59.910844 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:50:59.910800 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:50:59.911039 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:50:59.910908 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:00.913637 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:00.913606 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:00.914043 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:00.913615 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:00.914043 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:00.913696 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:00.914043 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:00.913775 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:01.910804 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:01.910753 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:01.910999 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:01.910872 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:02.910342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:02.910297 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:02.910832 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:02.910297 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:02.910832 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:02.910441 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:02.910832 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:02.910486 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:03.909967 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:03.909930 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:03.910230 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:03.910034 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:04.910355 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:04.910323 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:04.910833 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:04.910326 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:04.910833 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:04.910464 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:04.910833 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:04.910521 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:05.910703 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:05.910658 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:05.911069 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:05.910770 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:06.910593 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:06.910538 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:06.910765 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:06.910675 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:06.910765 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:06.910731 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:06.911061 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:06.910833 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:07.910956 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:07.910902 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:07.911357 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:07.911025 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:08.911303 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:08.911269 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:08.911693 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:08.911368 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:08.911693 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:08.911422 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:08.911693 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:08.911510 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:09.910328 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:09.910292 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:09.910529 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:09.910430 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:10.911008 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:10.910969 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:10.911453 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:10.910970 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:10.911453 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:10.911096 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:10.911453 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:10.911228 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:11.910863 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:11.910821 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:11.911064 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:11.911033 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:12.910488 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:12.910440 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:12.910746 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:12.910722 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:12.911417 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:12.911393 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:12.911772 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:12.911488 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:12.920480 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:12.920437 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:51:13.380969 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.380946 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/1.log" Sep 18 16:51:13.381369 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.381354 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/0.log" Sep 18 16:51:13.381419 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.381390 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" containerID="4cdce6e4f9cb5810455b90031ce756896d39c2751668f6efa02389004ec4190f" exitCode=1 Sep 18 16:51:13.381473 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.381457 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerDied","Data":"4cdce6e4f9cb5810455b90031ce756896d39c2751668f6efa02389004ec4190f"} Sep 18 16:51:13.381521 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.381511 2586 scope.go:117] "RemoveContainer" containerID="ac619034c78bcca7091c3424f793e9de568e8024dc8e2b352e285fcd578d94c2" Sep 18 16:51:13.381939 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.381922 2586 scope.go:117] "RemoveContainer" containerID="4cdce6e4f9cb5810455b90031ce756896d39c2751668f6efa02389004ec4190f" Sep 18 16:51:13.382104 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:13.382084 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 10s restarting failed container=kube-multus pod=multus-285nx_openshift-multus(1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3)\"" pod="openshift-multus/multus-285nx" podUID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" Sep 18 16:51:13.910503 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:13.910459 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:13.910704 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:13.910585 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:14.385456 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:14.385430 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/1.log" Sep 18 16:51:14.910299 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:14.910257 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:14.910501 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:14.910262 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:14.910501 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:14.910380 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:14.910501 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:14.910461 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:15.910016 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:15.909981 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:15.910390 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:15.910092 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:16.910876 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:16.910838 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:16.911359 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:16.910838 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:16.911359 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:16.910943 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:16.911359 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:16.911021 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:17.910967 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:17.910926 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:17.911327 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:17.911034 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:18.880541 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:18.880507 2586 kubelet_node_status.go:448] "Node not becoming ready in time after startup" Sep 18 16:51:18.894728 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:18.894688 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:18.911879 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:18.911847 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:18.912196 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:18.911977 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:18.912196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:18.912023 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:18.912196 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:18.912121 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:19.910746 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:19.910706 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:19.910933 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:19.910827 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:20.910809 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:20.910769 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:20.910809 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:20.910808 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:20.911227 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:20.910905 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:20.911227 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:20.910984 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:21.910389 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:21.910350 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:21.910583 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:21.910462 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:22.910376 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:22.910334 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:22.910783 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:22.910447 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:22.910783 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:22.910503 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:22.910783 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:22.910638 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:23.895849 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:23.895810 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:23.910132 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:23.910102 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:23.910238 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:23.910218 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:24.910813 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:24.910768 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:24.911277 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:24.910768 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:24.911277 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:24.910923 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:24.911277 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:24.910952 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:25.910745 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:25.910693 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:25.910931 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:25.910826 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:26.910320 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:26.910281 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:26.910498 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:26.910394 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:26.910498 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:26.910453 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:26.910634 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:26.910610 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:27.910248 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:27.910197 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:27.910717 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:27.910332 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:27.910717 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:27.910695 2586 scope.go:117] "RemoveContainer" containerID="4cdce6e4f9cb5810455b90031ce756896d39c2751668f6efa02389004ec4190f" Sep 18 16:51:28.421235 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:28.421157 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/1.log" Sep 18 16:51:28.421235 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:28.421207 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerStarted","Data":"9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478"} Sep 18 16:51:28.896169 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:28.896125 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:28.911723 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:28.911668 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:28.911723 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:28.911723 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:28.912180 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:28.911806 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:28.912180 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:28.911913 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:29.910613 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:29.910542 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:29.910774 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:29.910685 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:30.617979 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:30.617924 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:30.617979 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:30.617973 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:30.618413 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.618083 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:51:30.618413 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.618083 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:51:30.618413 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.618138 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:53:32.618123811 +0000 UTC m=+255.894248968 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:51:30.618413 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.618153 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:53:32.618145623 +0000 UTC m=+255.894270774 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:51:30.718269 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:30.718226 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:30.718406 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.718365 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:51:30.718406 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.718382 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:51:30.718406 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.718393 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:51:30.718495 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.718435 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:53:32.718424803 +0000 UTC m=+255.994549954 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:51:30.911050 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:30.910972 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:30.911164 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:30.910983 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:30.911164 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.911129 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:30.911240 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:30.911162 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:31.910409 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:31.910364 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:31.910836 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:31.910488 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:32.910444 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:32.910406 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:32.910881 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:32.910412 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:32.910881 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:32.910519 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:32.910881 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:32.910637 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:33.897510 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:33.897470 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:33.910827 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:33.910797 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:33.911128 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:33.910909 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:34.910985 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:34.910944 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:34.911502 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:34.910944 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:34.911502 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:34.911064 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:34.911502 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:34.911159 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:35.910876 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:35.910844 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:35.911045 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:35.910949 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:36.910707 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:36.910669 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:36.910906 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:36.910670 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:36.910906 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:36.910777 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:36.910906 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:36.910868 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:37.910613 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:37.910582 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:37.911047 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:37.910711 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:38.897789 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:38.897757 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:38.911767 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:38.911733 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:38.912143 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:38.911827 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:38.912143 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:38.911920 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:38.912143 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:38.912031 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:39.910519 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:39.910469 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:39.910753 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:39.910693 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:40.912979 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:40.912946 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:40.913377 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:40.913060 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:40.913377 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:40.913108 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:40.913377 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:40.913207 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:41.910358 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:41.910319 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:41.910578 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:41.910421 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:42.910405 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:42.910363 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:42.910405 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:42.910392 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:42.910975 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:42.910487 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:42.910975 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:42.910597 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:42.920201 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:42.920161 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:51:43.899155 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:43.899121 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:43.910410 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:43.910376 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:43.910550 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:43.910518 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:44.910770 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:44.910734 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:44.911183 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:44.910734 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:44.911183 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:44.910866 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:44.911183 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:44.910905 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:45.910351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:45.910319 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:45.910535 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:45.910422 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:46.910878 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:46.910840 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:46.911313 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:46.910855 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:46.911313 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:46.910956 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:46.911313 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:46.911076 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:47.464973 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:47.464943 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 16:51:47.467085 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:47.467056 2586 generic.go:356] "Generic (PLEG): container finished" podID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerID="59dea95c912039f6d64fdb5cec75eea0ddcaef7087cc14c7fd839ae6eb14e8d3" exitCode=1 Sep 18 16:51:47.467205 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:47.467107 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerDied","Data":"59dea95c912039f6d64fdb5cec75eea0ddcaef7087cc14c7fd839ae6eb14e8d3"} Sep 18 16:51:47.467837 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:47.467821 2586 scope.go:117] "RemoveContainer" containerID="59dea95c912039f6d64fdb5cec75eea0ddcaef7087cc14c7fd839ae6eb14e8d3" Sep 18 16:51:47.910841 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:47.910807 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:47.911003 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:47.910922 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:48.471342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:48.471318 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 16:51:48.473265 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:48.473242 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" event={"ID":"1297f04c-ff7b-4c98-a6ee-9fa6b887a561","Type":"ContainerStarted","Data":"d4bdd0f74968c5c36fc11189d0a7abeb77ecce04a4eaadad15a5e2ed06a0a893"} Sep 18 16:51:48.473753 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:48.473734 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:51:48.899511 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:48.899467 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:48.911292 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:48.911260 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:48.911596 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:48.911370 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:48.911596 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:48.911412 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:48.911596 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:48.911506 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:49.910419 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:49.910370 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:49.910617 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:49.910487 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:50.910744 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:50.910707 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:50.911219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:50.910708 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:50.911219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:50.910823 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:50.911219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:50.910884 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:51.910716 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:51.910668 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:51.910894 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:51.910793 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:52.910691 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:52.910653 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:52.910891 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:52.910811 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:52.910891 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:52.910858 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:52.911254 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:52.910974 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:53.900238 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:53.900201 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:53.910487 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:53.910458 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:53.910613 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:53.910594 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:54.910836 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:54.910796 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:54.911193 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:54.910796 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:54.911193 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:54.910920 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:54.911193 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:54.910985 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:55.910896 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:55.910843 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:55.911289 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:55.910973 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:56.910999 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:56.910948 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:56.911478 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:56.910948 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:56.911478 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:56.911081 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:56.911478 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:56.911138 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:57.910636 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:57.910600 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:57.910811 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:57.910781 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:51:58.900669 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:58.900587 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:51:58.911949 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:58.911909 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:51:58.912135 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:58.912019 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:51:58.912135 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:58.912070 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:51:58.912225 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:58.912165 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:51:59.911030 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:51:59.910975 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:51:59.911430 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:51:59.911107 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:00.910890 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:00.910840 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:00.911073 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:00.910906 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:00.911073 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:00.911035 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:00.911387 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:00.911167 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:01.910408 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:01.910355 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:01.910628 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:01.910484 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:02.910167 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:02.910121 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:02.910167 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:02.910149 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:02.910632 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:02.910241 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:02.910632 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:02.910324 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:03.902213 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:03.902178 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:03.910667 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:03.910642 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:03.910965 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:03.910745 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:04.913334 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:04.913308 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:04.913713 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:04.913409 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:04.913713 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:04.913483 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:04.913713 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:04.913584 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:05.910162 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:05.910112 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:05.910338 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:05.910239 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:06.913336 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:06.913305 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:06.913814 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:06.913402 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:06.913814 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:06.913451 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:06.913814 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:06.913566 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:07.910749 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:07.910713 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:07.910927 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:07.910816 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:08.902587 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:08.902532 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:08.911400 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:08.911372 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:08.911511 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:08.911471 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:08.911583 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:08.911518 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:08.911650 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:08.911631 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:09.909985 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:09.909936 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:09.910387 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:09.910071 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:10.913683 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:10.913647 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:10.913683 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:10.913671 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:10.914117 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:10.913649 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:10.914117 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:10.913736 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:10.914117 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:10.913818 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:10.914117 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:10.913877 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:12.910270 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:12.910236 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:12.910703 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:12.910237 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:12.910703 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:12.910346 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:12.910703 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:12.910252 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:12.910703 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:12.910427 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:12.910703 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:12.910511 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:13.534821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:13.534744 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/2.log" Sep 18 16:52:13.535165 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:13.535149 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/1.log" Sep 18 16:52:13.535264 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:13.535188 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" containerID="9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478" exitCode=1 Sep 18 16:52:13.535264 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:13.535224 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerDied","Data":"9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478"} Sep 18 16:52:13.535444 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:13.535285 2586 scope.go:117] "RemoveContainer" containerID="4cdce6e4f9cb5810455b90031ce756896d39c2751668f6efa02389004ec4190f" Sep 18 16:52:13.535699 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:13.535676 2586 scope.go:117] "RemoveContainer" containerID="9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478" Sep 18 16:52:13.535862 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:13.535843 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 20s restarting failed container=kube-multus pod=multus-285nx_openshift-multus(1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3)\"" pod="openshift-multus/multus-285nx" podUID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" Sep 18 16:52:13.903354 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:13.903317 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:14.538704 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:14.538677 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/2.log" Sep 18 16:52:14.910573 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:14.910511 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:14.910573 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:14.910525 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:14.910573 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:14.910525 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:14.910876 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:14.910747 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:14.910930 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:14.910899 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:14.910972 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:14.910935 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:16.911042 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:16.911000 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:16.911522 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:16.911008 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:16.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:16.911131 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:16.911522 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:16.911020 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:16.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:16.911199 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:16.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:16.911315 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:18.903736 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:18.903698 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:18.911544 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:18.911510 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:18.911721 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:18.911649 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:18.911721 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:18.911660 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:18.911860 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:18.911647 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:18.911860 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:18.911762 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:18.911860 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:18.911818 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:19.486300 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:19.486262 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:52:20.910242 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:20.910199 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:20.910711 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:20.910199 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:20.910711 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:20.910199 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:20.910711 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:20.910440 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:20.910711 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:20.910500 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:20.910711 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:20.910335 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:22.910762 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:22.910720 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:22.911248 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:22.910719 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:22.911248 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:22.910840 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:22.911248 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:22.910731 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:22.911248 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:22.910980 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:22.911248 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:22.911038 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:23.904340 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:23.904303 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:24.910800 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:24.910762 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:24.910800 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:24.910766 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:24.911408 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:24.910771 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:24.911408 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:24.910994 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:24.911408 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:24.911128 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:24.911408 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:24.911162 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:26.910786 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:26.910749 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:26.911170 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:26.910749 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:26.911170 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:26.910868 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:26.911170 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:26.910753 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:26.911170 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:26.910967 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:26.911170 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:26.911024 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:26.911353 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:26.911243 2586 scope.go:117] "RemoveContainer" containerID="9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478" Sep 18 16:52:26.911399 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:26.911360 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 20s restarting failed container=kube-multus pod=multus-285nx_openshift-multus(1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3)\"" pod="openshift-multus/multus-285nx" podUID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" Sep 18 16:52:28.904706 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:28.904663 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:28.910770 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:28.910741 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:28.910941 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:28.910851 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:28.911113 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:28.911080 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:28.911219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:28.911172 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:28.911271 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:28.911227 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:28.911338 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:28.911314 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:30.910265 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:30.910228 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:30.910786 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:30.910230 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:30.910786 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:30.910344 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:30.910786 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:30.910237 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:30.910786 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:30.910424 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:30.910786 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:30.910474 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:32.910760 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:32.910719 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:32.910760 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:32.910742 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:32.911334 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:32.910742 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:32.911334 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:32.910828 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:32.911334 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:32.910918 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:32.911334 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:32.910983 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:33.905972 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:33.905934 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:34.910174 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:34.910136 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:34.910174 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:34.910165 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:34.910752 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:34.910264 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:34.910752 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:34.910289 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:34.910752 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:34.910359 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:34.910752 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:34.910438 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:36.910745 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:36.910708 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:36.911132 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:36.910775 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:36.911132 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:36.910781 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:36.911132 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:36.911053 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:36.911277 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:36.911158 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:36.911277 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:36.911219 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:38.906351 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:38.906307 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:38.911014 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:38.910983 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:38.911158 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:38.911070 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:38.911158 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:38.911137 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:38.911280 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:38.911238 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:38.911337 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:38.911311 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:38.911424 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:38.911405 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:40.910191 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:40.910158 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:40.910713 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:40.910158 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:40.910713 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:40.910270 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:40.910713 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:40.910158 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:40.910713 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:40.910342 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:40.910713 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:40.910406 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:41.910534 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:41.910504 2586 scope.go:117] "RemoveContainer" containerID="9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478" Sep 18 16:52:42.606303 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:42.606274 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/2.log" Sep 18 16:52:42.606663 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:42.606636 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerStarted","Data":"73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf"} Sep 18 16:52:42.910081 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:42.909998 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:42.910254 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:42.909998 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:42.910254 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:42.910118 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:42.910254 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:42.909998 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:42.910254 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:42.910180 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:42.910416 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:42.910266 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:43.907135 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:43.907098 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:44.910287 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:44.910245 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:44.910791 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:44.910243 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:44.910791 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:44.910356 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:44.910791 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:44.910461 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:44.910791 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:44.910253 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:44.910791 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:44.910597 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:46.910574 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:46.910518 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:46.910941 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:46.910517 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:46.910941 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:46.910651 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:46.910941 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:46.910753 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:46.910941 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:46.910515 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:46.910941 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:46.910850 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:48.907460 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:48.907418 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:48.910901 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:48.910878 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:48.911060 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:48.910965 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:48.911060 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:48.910980 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:48.911172 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:48.911058 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:48.911172 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:48.911087 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:48.911237 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:48.911211 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:49.486417 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:49.486379 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:52:50.910851 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:50.910802 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:50.910851 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:50.910845 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:50.911267 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:50.910802 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:50.911267 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:50.910939 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:50.911267 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:50.911043 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:50.911267 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:50.911110 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:52.910101 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:52.910059 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:52.910530 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:52.910059 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:52.910530 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:52.910171 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:52.910530 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:52.910059 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:52.910530 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:52.910292 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:52.910530 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:52.910344 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:53.908064 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:53.908026 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:54.910873 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:54.910833 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:54.911259 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:54.910846 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:54.911259 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:54.910952 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:54.911259 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:54.910846 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:54.911259 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:54.911022 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:54.911259 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:54.911107 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:56.910848 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:56.910811 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:56.910848 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:56.910841 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:56.911346 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:56.910822 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:56.911346 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:56.910920 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:52:56.911346 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:56.911001 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:56.911346 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:56.911057 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:58.908397 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:58.908357 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:52:58.912312 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:58.912281 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:52:58.912686 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:58.912659 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:52:58.913053 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:58.913036 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:52:58.913323 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:58.913302 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:52:58.913630 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:52:58.913614 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:52:58.913835 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:52:58.913816 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:00.910859 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:00.910821 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:00.911340 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:00.910821 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:00.911340 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:00.910965 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:00.911340 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:00.910833 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:00.911340 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:00.911009 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:00.911340 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:00.911104 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:01.920921 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:01.920883 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:53:02.910772 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:02.910736 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:02.910975 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:02.910736 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:02.910975 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:02.910838 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:02.910975 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:02.910751 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:02.910975 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:02.910909 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:02.910975 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:02.910970 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:03.909343 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:03.909160 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:04.910089 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:04.910046 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:04.910594 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:04.910046 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:04.910594 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:04.910157 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:04.910594 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:04.910249 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:04.910594 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:04.910046 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:04.910594 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:04.910334 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:06.910175 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:06.910139 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:06.910653 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:06.910149 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:06.910653 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:06.910276 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:06.910653 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:06.910154 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:06.910653 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:06.910313 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:06.910653 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:06.910404 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:08.909837 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:08.909803 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:08.910982 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:08.910950 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:08.911109 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:08.911073 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:08.911165 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:08.911102 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:08.911165 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:08.911118 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:08.911244 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:08.911208 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:08.911313 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:08.911286 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:10.910458 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:10.910424 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:10.910458 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:10.910456 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:10.910963 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:10.910428 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:10.910963 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:10.910548 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:10.910963 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:10.910638 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:10.910963 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:10.910690 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:12.910293 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:12.910252 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:12.910810 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:12.910252 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:12.910810 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:12.910370 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:12.910810 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:12.910257 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:12.910810 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:12.910433 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:12.910810 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:12.910506 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:13.911056 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:13.911009 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:14.913394 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:14.913364 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:14.913887 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:14.913466 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:14.913887 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:14.913460 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:14.913887 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:14.913683 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:14.913887 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:14.913676 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:14.913887 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:14.913737 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:16.910925 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:16.910884 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:16.910925 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:16.910906 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:16.911476 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:16.910994 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:16.911476 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:16.911023 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:16.911476 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:16.911091 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:16.911476 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:16.911160 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:18.911344 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:18.911297 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:18.911804 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:18.911531 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:18.911804 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:18.911648 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:18.911804 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:18.911680 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:18.911804 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:18.911692 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:18.911804 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:18.911753 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:18.912049 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:18.911835 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:20.911049 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:20.911004 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:20.911408 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:20.911010 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:20.911408 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:20.911159 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:20.911408 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:20.911010 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:20.911408 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:20.911198 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:20.911408 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:20.911273 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:22.910720 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:22.910673 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:22.910720 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:22.910716 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:22.911245 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:22.910700 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:22.911245 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:22.910802 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:22.911245 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:22.910956 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:22.911245 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:22.911055 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:23.912694 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:23.912659 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:24.910276 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:24.910241 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:24.910463 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:24.910247 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:24.910463 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:24.910357 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:24.910463 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:24.910247 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:24.910666 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:24.910423 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:24.910666 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:24.910527 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:26.910311 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:26.910272 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:26.910811 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:26.910316 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:26.910811 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:26.910286 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:26.910811 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:26.910408 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:26.910811 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:26.910466 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:26.910811 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:26.910545 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:27.713379 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:27.713348 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:53:27.713803 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:27.713787 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/2.log" Sep 18 16:53:27.713880 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:27.713821 2586 generic.go:356] "Generic (PLEG): container finished" podID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" containerID="73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf" exitCode=1 Sep 18 16:53:27.713920 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:27.713879 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerDied","Data":"73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf"} Sep 18 16:53:27.713965 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:27.713926 2586 scope.go:117] "RemoveContainer" containerID="9d18bf3bc27682f77efb338c528828bfa70642a34c9f5aa3524f371dadd95478" Sep 18 16:53:27.714287 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:27.714253 2586 scope.go:117] "RemoveContainer" containerID="73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf" Sep 18 16:53:27.714441 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:27.714419 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 40s restarting failed container=kube-multus pod=multus-285nx_openshift-multus(1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3)\"" pod="openshift-multus/multus-285nx" podUID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" Sep 18 16:53:28.717921 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:28.717853 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:53:28.912101 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:28.912071 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:28.912262 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:28.912169 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:28.912262 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:28.912210 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:28.912348 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:28.912273 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:28.912348 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:28.912283 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:28.912434 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:28.912349 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:28.913129 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:28.913098 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:30.910597 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:30.910537 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:30.910597 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:30.910583 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:30.911096 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:30.910597 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:30.911096 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:30.910685 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:30.911096 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:30.910752 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:30.911096 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:30.910849 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:31.921301 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:31.921261 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" podUID="1297f04c-ff7b-4c98-a6ee-9fa6b887a561" containerName="ovnkube-controller" probeResult="failure" output="" Sep 18 16:53:32.689067 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:32.689012 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:32.689067 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:32.689067 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:32.689292 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.689174 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 18 16:53:32.689292 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.689180 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:53:32.689292 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.689223 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret podName:b2aace0d-8144-4803-b013-63eeaf617aca nodeName:}" failed. No retries permitted until 2026-09-18 16:55:34.689210635 +0000 UTC m=+377.965335787 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret") pod "global-pull-secret-syncer-9hg2q" (UID: "b2aace0d-8144-4803-b013-63eeaf617aca") : object "kube-system"/"original-pull-secret" not registered Sep 18 16:53:32.689292 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.689237 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs podName:776f65d4-9fed-4694-8139-fb8bb9a95c7b nodeName:}" failed. No retries permitted until 2026-09-18 16:55:34.689230305 +0000 UTC m=+377.965355457 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs") pod "network-metrics-daemon-5jqgt" (UID: "776f65d4-9fed-4694-8139-fb8bb9a95c7b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 18 16:53:32.790306 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:32.790258 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:32.790475 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.790420 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 18 16:53:32.790475 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.790438 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 18 16:53:32.790475 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.790448 2586 projected.go:196] Error preparing data for projected volume kube-api-access-z7nzj for pod openshift-network-diagnostics/network-check-target-kld7t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:53:32.790603 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.790493 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj podName:2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f nodeName:}" failed. No retries permitted until 2026-09-18 16:55:34.790481224 +0000 UTC m=+378.066606374 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z7nzj" (UniqueName: "kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj") pod "network-check-target-kld7t" (UID: "2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 18 16:53:32.912961 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:32.912929 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:32.913129 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.913021 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:32.913129 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:32.913058 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:32.913129 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.913107 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:32.913129 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:32.913131 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:32.913251 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:32.913172 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:33.914672 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:33.914632 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:34.910918 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:34.910879 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:34.911086 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:34.910883 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:34.911086 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:34.910992 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:34.911158 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:34.911085 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:34.911158 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:34.910884 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:34.911229 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:34.911194 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:36.910859 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:36.910818 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:36.911224 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:36.910822 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:36.911224 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:36.910964 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:36.911224 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:36.910818 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:36.911224 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:36.911011 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:36.911224 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:36.911087 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:38.911383 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:38.911351 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:38.911865 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:38.911434 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:38.911865 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:38.911610 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:38.911865 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:38.911643 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:38.911865 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:38.911763 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:38.912092 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:38.911909 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:38.915001 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:38.914973 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:40.910738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:40.910700 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:40.911219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:40.910700 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:40.911219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:40.910824 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:40.911219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:40.910882 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:40.911219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:40.910715 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:40.911219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:40.910942 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:41.910661 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:41.910621 2586 scope.go:117] "RemoveContainer" containerID="73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf" Sep 18 16:53:41.910949 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:41.910815 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 40s restarting failed container=kube-multus pod=multus-285nx_openshift-multus(1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3)\"" pod="openshift-multus/multus-285nx" podUID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" Sep 18 16:53:42.910098 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:42.910050 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:42.910267 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:42.910050 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:42.910267 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:42.910173 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:42.910267 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:42.910058 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:42.910369 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:42.910281 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:42.910369 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:42.910332 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:43.915846 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:43.915809 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:44.722441 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:44.722404 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9hg2q"] Sep 18 16:53:44.722643 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:44.722527 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:44.722722 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:44.722653 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:44.724665 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:44.724530 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kld7t"] Sep 18 16:53:44.724665 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:44.724646 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:44.724850 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:44.724747 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:44.738977 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:44.738945 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5jqgt"] Sep 18 16:53:44.739136 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:44.739041 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:44.739201 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:44.739134 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:45.911161 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:45.910966 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:45.911885 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:45.911257 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:46.910428 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:46.910387 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:46.910631 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:46.910510 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:46.910631 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:46.910387 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:46.911001 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:46.910979 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:47.910901 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:47.910859 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:47.911293 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:47.911004 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:48.913269 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:48.913236 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:48.913715 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:48.913360 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:48.913715 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:48.913361 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:48.913715 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:48.913428 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:48.916133 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:48.916105 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:49.910042 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:49.910004 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:49.910232 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:49.910119 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:50.910911 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:50.910876 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:50.910911 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:50.910891 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:50.911348 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:50.911157 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:50.911348 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:50.911291 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:51.910812 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:51.910765 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:51.910999 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:51.910887 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:52.910270 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:52.910232 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:52.910428 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:52.910232 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:52.910428 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:52.910358 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:52.910428 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:52.910411 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:53.910971 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.910933 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:53.911356 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:53.911049 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:53.917500 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:53.917467 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:53.937808 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.937780 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-954n4"] Sep 18 16:53:53.940875 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.940859 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:53.944515 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.944489 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 18 16:53:53.944668 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.944496 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 18 16:53:53.945179 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.945161 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 18 16:53:53.945250 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.945178 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 18 16:53:53.945747 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.945600 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 18 16:53:53.945747 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.945613 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kslbc\"" Sep 18 16:53:53.945747 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:53.945640 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 18 16:53:54.024956 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.024916 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-sys\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.024956 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.024955 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025008 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-wtmp\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025027 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-tls\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025040 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7707d9e-e512-4252-b07a-e26a44bf885c-metrics-client-ca\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025056 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-root\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025075 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-textfile\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tws2g\" (UniqueName: \"kubernetes.io/projected/f7707d9e-e512-4252-b07a-e26a44bf885c-kube-api-access-tws2g\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.025196 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.025147 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-accelerators-collector-config\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125705 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7707d9e-e512-4252-b07a-e26a44bf885c-metrics-client-ca\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125741 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-root\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125761 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-textfile\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125775 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tws2g\" (UniqueName: \"kubernetes.io/projected/f7707d9e-e512-4252-b07a-e26a44bf885c-kube-api-access-tws2g\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125791 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-accelerators-collector-config\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125814 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-sys\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125831 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-root\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125839 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.125897 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125890 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-sys\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.126219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125965 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-wtmp\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.126219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.125996 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-tls\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.126219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:54.126081 2586 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 18 16:53:54.126219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.126144 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-wtmp\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.126219 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:54.126155 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-tls podName:f7707d9e-e512-4252-b07a-e26a44bf885c nodeName:}" failed. No retries permitted until 2026-09-18 16:53:54.626136346 +0000 UTC m=+277.902261496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-tls") pod "node-exporter-954n4" (UID: "f7707d9e-e512-4252-b07a-e26a44bf885c") : secret "node-exporter-tls" not found Sep 18 16:53:54.126219 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.126151 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-textfile\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.126500 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.126355 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7707d9e-e512-4252-b07a-e26a44bf885c-metrics-client-ca\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.126500 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.126458 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-accelerators-collector-config\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.130034 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.130007 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.147725 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.147689 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tws2g\" (UniqueName: \"kubernetes.io/projected/f7707d9e-e512-4252-b07a-e26a44bf885c-kube-api-access-tws2g\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.630242 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.630212 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-tls\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.632671 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.632655 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f7707d9e-e512-4252-b07a-e26a44bf885c-node-exporter-tls\") pod \"node-exporter-954n4\" (UID: \"f7707d9e-e512-4252-b07a-e26a44bf885c\") " pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.854480 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.854446 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-954n4" Sep 18 16:53:54.866119 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:53:54.866064 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7707d9e_e512_4252_b07a_e26a44bf885c.slice/crio-6eddf1146728040f161876d469e589ff6cf1e3ef021976aa57150fb7a344ed92 WatchSource:0}: Error finding container 6eddf1146728040f161876d469e589ff6cf1e3ef021976aa57150fb7a344ed92: Status 404 returned error can't find the container with id 6eddf1146728040f161876d469e589ff6cf1e3ef021976aa57150fb7a344ed92 Sep 18 16:53:54.910800 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.910727 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:54.910800 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:54.910780 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:54.910945 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:54.910865 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:54.910985 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:54.910961 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:55.783204 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:55.783166 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-954n4" event={"ID":"f7707d9e-e512-4252-b07a-e26a44bf885c","Type":"ContainerStarted","Data":"6eddf1146728040f161876d469e589ff6cf1e3ef021976aa57150fb7a344ed92"} Sep 18 16:53:55.910681 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:55.910658 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:55.910779 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:55.910761 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:55.910830 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:55.910809 2586 scope.go:117] "RemoveContainer" containerID="73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf" Sep 18 16:53:55.910966 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:55.910949 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"kube-multus\" with CrashLoopBackOff: \"back-off 40s restarting failed container=kube-multus pod=multus-285nx_openshift-multus(1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3)\"" pod="openshift-multus/multus-285nx" podUID="1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3" Sep 18 16:53:56.787245 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:56.787053 2586 generic.go:356] "Generic (PLEG): container finished" podID="f7707d9e-e512-4252-b07a-e26a44bf885c" containerID="d8d48185d806a9bf1521caa0c270de82be2f807ef0f08bd0bab3947d40dabb53" exitCode=0 Sep 18 16:53:56.787694 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:56.787140 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-954n4" event={"ID":"f7707d9e-e512-4252-b07a-e26a44bf885c","Type":"ContainerDied","Data":"d8d48185d806a9bf1521caa0c270de82be2f807ef0f08bd0bab3947d40dabb53"} Sep 18 16:53:56.910497 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:56.910471 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:56.910497 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:56.910490 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:56.910660 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:56.910601 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:56.910769 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:56.910745 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:57.793994 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:57.793958 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-954n4" event={"ID":"f7707d9e-e512-4252-b07a-e26a44bf885c","Type":"ContainerStarted","Data":"26ea11ec553fde9c382f1cdf5bbe02d1a8620744a578994f81b0edb23622fcbd"} Sep 18 16:53:57.794388 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:57.794011 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-954n4" event={"ID":"f7707d9e-e512-4252-b07a-e26a44bf885c","Type":"ContainerStarted","Data":"1125def6ae0448d23f9cf1be2345ec5dae55fdfd80d270c5a46ff53c1d52e878"} Sep 18 16:53:57.910327 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:57.910289 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:57.910473 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:57.910410 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:53:58.911500 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:58.911454 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:53:58.911960 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:58.911595 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:53:58.911960 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:58.911648 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:53:58.911960 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:58.911758 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:53:58.917879 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:58.917841 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:53:59.910303 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:53:59.910243 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:53:59.910495 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:53:59.910402 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:00.910646 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:00.910613 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:00.911125 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:00.910613 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:00.911125 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:00.910725 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:00.911125 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:00.910797 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:01.910233 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:01.910198 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:01.910434 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:01.910335 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:01.920958 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:01.920930 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8hzb9" Sep 18 16:54:01.951836 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:01.951785 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-954n4" podStartSLOduration=7.869769382 podStartE2EDuration="8.951768698s" totalImagesPullingTime="1.008221188s" totalInitContainerRuntime="73.778128ms" isStatefulPod=true podCreationTimestamp="2026-09-18 16:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:53:57.821802124 +0000 UTC m=+281.097927297" watchObservedRunningTime="2026-09-18 16:54:01.951768698 +0000 UTC m=+285.227893868" Sep 18 16:54:02.910677 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:02.910641 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:02.910859 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:02.910641 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:02.910859 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:02.910764 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:02.910859 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:02.910813 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:03.910493 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:03.910454 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:03.910977 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:03.910603 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:03.918409 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:03.918381 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:54:04.911042 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:04.910994 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:04.911522 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:04.911044 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:04.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:04.911157 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:04.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:04.911252 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:05.910916 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:05.910876 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:05.911103 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:05.911019 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:06.910821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:06.910776 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:06.910821 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:06.910807 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:06.911092 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:06.910909 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:06.911092 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:06.910998 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:07.910677 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:07.910642 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:07.910870 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:07.910764 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:08.911059 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:08.911018 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:08.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:08.911123 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:08.911522 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:08.911135 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:08.911522 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:08.911218 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:08.911672 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:08.911657 2586 scope.go:117] "RemoveContainer" containerID="73388e9b6ebbe72870c94d9db86f49109622978f68a14bc553ae8a0d505a64cf" Sep 18 16:54:08.918802 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:08.918771 2586 kubelet.go:3032] "Container runtime network not ready" networkReady="NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" Sep 18 16:54:09.825574 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:09.825527 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:54:09.825745 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:09.825595 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-285nx" event={"ID":"1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3","Type":"ContainerStarted","Data":"08c3fe95065c9963d343767e7746aafb1540e16cac7e844f17adddad5e712138"} Sep 18 16:54:09.910869 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:09.910824 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:09.911068 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:09.910966 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:10.910829 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:10.910793 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:10.911580 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:10.910795 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:10.911580 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:10.910902 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:10.911580 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:10.910975 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:11.910632 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:11.910596 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:11.910835 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:11.910735 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:12.910480 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:12.910444 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:12.910960 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:12.910455 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:12.910960 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:12.910549 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9hg2q" podUID="b2aace0d-8144-4803-b013-63eeaf617aca" Sep 18 16:54:12.910960 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:12.910640 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kld7t" podUID="2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f" Sep 18 16:54:13.910617 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:13.910577 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:13.911077 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:54:13.910690 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5jqgt" podUID="776f65d4-9fed-4694-8139-fb8bb9a95c7b" Sep 18 16:54:14.910738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.910697 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:54:14.910738 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.910724 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:54:14.913952 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.913928 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 16:54:14.914066 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.913980 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 16:54:14.915051 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.915030 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7dj2b\"" Sep 18 16:54:14.915120 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.915040 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 16:54:14.961648 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:14.961619 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-188.ec2.internal" event="NodeReady" Sep 18 16:54:15.071101 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.071067 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dn64f"] Sep 18 16:54:15.073937 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.073911 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.074451 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.074433 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b64vj"] Sep 18 16:54:15.076542 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.076519 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4lvzz\"" Sep 18 16:54:15.076644 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.076621 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 18 16:54:15.076713 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.076659 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 18 16:54:15.077126 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.077112 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.079745 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.079724 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 18 16:54:15.079884 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.079862 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8qvq5\"" Sep 18 16:54:15.079960 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.079945 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 18 16:54:15.080028 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.080012 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 18 16:54:15.080149 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.080054 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 18 16:54:15.088600 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.088549 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dn64f"] Sep 18 16:54:15.089278 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.089254 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b64vj"] Sep 18 16:54:15.167484 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167383 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/043c6094-46a5-46ea-998d-964d015ed767-data-volume\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.167484 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167448 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4196f06f-d002-4987-a299-3e65a0e22677-config-volume\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.167484 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167475 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5c5h\" (UniqueName: \"kubernetes.io/projected/4196f06f-d002-4987-a299-3e65a0e22677-kube-api-access-d5c5h\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.167823 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167590 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/043c6094-46a5-46ea-998d-964d015ed767-crio-socket\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.167823 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167632 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn4js\" (UniqueName: \"kubernetes.io/projected/043c6094-46a5-46ea-998d-964d015ed767-kube-api-access-nn4js\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.167823 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167679 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/043c6094-46a5-46ea-998d-964d015ed767-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.167823 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167742 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4196f06f-d002-4987-a299-3e65a0e22677-metrics-tls\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.167823 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167777 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/043c6094-46a5-46ea-998d-964d015ed767-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.167823 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.167810 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4196f06f-d002-4987-a299-3e65a0e22677-tmp-dir\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.268321 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268285 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/043c6094-46a5-46ea-998d-964d015ed767-data-volume\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.268462 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268343 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4196f06f-d002-4987-a299-3e65a0e22677-config-volume\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.268462 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268359 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5c5h\" (UniqueName: \"kubernetes.io/projected/4196f06f-d002-4987-a299-3e65a0e22677-kube-api-access-d5c5h\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.268595 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268530 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/043c6094-46a5-46ea-998d-964d015ed767-crio-socket\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.268657 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268592 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn4js\" (UniqueName: \"kubernetes.io/projected/043c6094-46a5-46ea-998d-964d015ed767-kube-api-access-nn4js\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.268657 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268627 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/043c6094-46a5-46ea-998d-964d015ed767-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.268762 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268657 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/043c6094-46a5-46ea-998d-964d015ed767-crio-socket\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.268762 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268669 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4196f06f-d002-4987-a299-3e65a0e22677-metrics-tls\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.268762 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268728 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/043c6094-46a5-46ea-998d-964d015ed767-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.268762 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268759 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4196f06f-d002-4987-a299-3e65a0e22677-tmp-dir\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.268928 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.268812 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/043c6094-46a5-46ea-998d-964d015ed767-data-volume\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.269076 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.269057 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4196f06f-d002-4987-a299-3e65a0e22677-config-volume\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.269152 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.269135 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4196f06f-d002-4987-a299-3e65a0e22677-tmp-dir\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.269272 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.269235 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/043c6094-46a5-46ea-998d-964d015ed767-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.272277 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.272252 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/043c6094-46a5-46ea-998d-964d015ed767-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.272408 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.272293 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4196f06f-d002-4987-a299-3e65a0e22677-metrics-tls\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.278243 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.278203 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn4js\" (UniqueName: \"kubernetes.io/projected/043c6094-46a5-46ea-998d-964d015ed767-kube-api-access-nn4js\") pod \"insights-runtime-extractor-b64vj\" (UID: \"043c6094-46a5-46ea-998d-964d015ed767\") " pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.278385 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.278367 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5c5h\" (UniqueName: \"kubernetes.io/projected/4196f06f-d002-4987-a299-3e65a0e22677-kube-api-access-d5c5h\") pod \"dns-default-dn64f\" (UID: \"4196f06f-d002-4987-a299-3e65a0e22677\") " pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.383703 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.383665 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:15.389040 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.389023 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b64vj" Sep 18 16:54:15.678498 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.678459 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dn64f"] Sep 18 16:54:15.680036 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.680008 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b64vj"] Sep 18 16:54:15.683989 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:54:15.683961 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4196f06f_d002_4987_a299_3e65a0e22677.slice/crio-1698dc9cc04c0594088c7c21bd0ea7c44137700acfccdcae6d4a068a66dc0521 WatchSource:0}: Error finding container 1698dc9cc04c0594088c7c21bd0ea7c44137700acfccdcae6d4a068a66dc0521: Status 404 returned error can't find the container with id 1698dc9cc04c0594088c7c21bd0ea7c44137700acfccdcae6d4a068a66dc0521 Sep 18 16:54:15.684544 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:54:15.684444 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043c6094_46a5_46ea_998d_964d015ed767.slice/crio-8f0074d820d91931a53b1b9afa470fc481058c35b864e66daff4027e4ce35010 WatchSource:0}: Error finding container 8f0074d820d91931a53b1b9afa470fc481058c35b864e66daff4027e4ce35010: Status 404 returned error can't find the container with id 8f0074d820d91931a53b1b9afa470fc481058c35b864e66daff4027e4ce35010 Sep 18 16:54:15.842911 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.842878 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b64vj" event={"ID":"043c6094-46a5-46ea-998d-964d015ed767","Type":"ContainerStarted","Data":"5c3ba28bfaea68e5675f37e7f5454bdca856455006a821506dccef293f6e2bed"} Sep 18 16:54:15.843092 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.842934 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b64vj" event={"ID":"043c6094-46a5-46ea-998d-964d015ed767","Type":"ContainerStarted","Data":"8f0074d820d91931a53b1b9afa470fc481058c35b864e66daff4027e4ce35010"} Sep 18 16:54:15.844025 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.843998 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dn64f" event={"ID":"4196f06f-d002-4987-a299-3e65a0e22677","Type":"ContainerStarted","Data":"1698dc9cc04c0594088c7c21bd0ea7c44137700acfccdcae6d4a068a66dc0521"} Sep 18 16:54:15.910691 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.910640 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:54:15.913475 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.913452 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 16:54:15.913912 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:15.913510 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rqwhk\"" Sep 18 16:54:16.849450 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:16.849192 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b64vj" event={"ID":"043c6094-46a5-46ea-998d-964d015ed767","Type":"ContainerStarted","Data":"64021916bcfdc484b79a9a7d4ac77441708ae19c5bfece41a3163273103ce33d"} Sep 18 16:54:18.791073 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.791047 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:54:18.796814 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.796783 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 16:54:18.800863 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.800838 2586 kubelet.go:1648] "Image garbage collection succeeded" Sep 18 16:54:18.818041 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.818007 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:54:18.819992 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.819973 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 16:54:18.857774 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.857735 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b64vj" event={"ID":"043c6094-46a5-46ea-998d-964d015ed767","Type":"ContainerStarted","Data":"8710af5aef7e104af168221db21de85581711e0ebc0842966a7eeb38d630f4a0"} Sep 18 16:54:18.859460 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.859431 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dn64f" event={"ID":"4196f06f-d002-4987-a299-3e65a0e22677","Type":"ContainerStarted","Data":"7850f5fd79c0608e2c949950e344a5bc6db2ac3402889f41379cfbe1c6635dba"} Sep 18 16:54:18.859651 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.859484 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dn64f" event={"ID":"4196f06f-d002-4987-a299-3e65a0e22677","Type":"ContainerStarted","Data":"329940b8d463ee5a32ef9ba7c5366a36228a08c03cfa0e35accaff9e1b031aa5"} Sep 18 16:54:18.859651 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.859532 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dn64f" Sep 18 16:54:18.876992 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.876836 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b64vj" podStartSLOduration=1.7808593670000001 podStartE2EDuration="3.876819328s" totalImagesPullingTime="2.095959961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:54:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:54:18.876064683 +0000 UTC m=+302.152189868" watchObservedRunningTime="2026-09-18 16:54:18.876819328 +0000 UTC m=+302.152944500" Sep 18 16:54:18.895160 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:18.895118 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dn64f" podStartSLOduration=1.664539295 podStartE2EDuration="3.895100135s" totalImagesPullingTime="2.23056084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:54:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:54:18.89424683 +0000 UTC m=+302.170372026" watchObservedRunningTime="2026-09-18 16:54:18.895100135 +0000 UTC m=+302.171225308" Sep 18 16:54:28.864894 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:54:28.864852 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dn64f" Sep 18 16:55:34.746424 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.746375 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:55:34.746424 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.746420 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:55:34.750342 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.750315 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 18 16:55:34.750488 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.750316 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 18 16:55:34.761277 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.761244 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b2aace0d-8144-4803-b013-63eeaf617aca-original-pull-secret\") pod \"global-pull-secret-syncer-9hg2q\" (UID: \"b2aace0d-8144-4803-b013-63eeaf617aca\") " pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:55:34.761277 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.761263 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/776f65d4-9fed-4694-8139-fb8bb9a95c7b-metrics-certs\") pod \"network-metrics-daemon-5jqgt\" (UID: \"776f65d4-9fed-4694-8139-fb8bb9a95c7b\") " pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:55:34.823052 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.823018 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rqwhk\"" Sep 18 16:55:34.830605 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.830581 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5jqgt" Sep 18 16:55:34.847817 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.847787 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:55:34.850785 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.850767 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 18 16:55:34.860712 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.860684 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 18 16:55:34.871670 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:34.871636 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7nzj\" (UniqueName: \"kubernetes.io/projected/2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f-kube-api-access-z7nzj\") pod \"network-check-target-kld7t\" (UID: \"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f\") " pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:55:35.024986 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.024899 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7dj2b\"" Sep 18 16:55:35.025963 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.025942 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9hg2q" Sep 18 16:55:35.032853 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.032830 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:55:35.109425 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.109390 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5jqgt"] Sep 18 16:55:35.113713 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:55:35.113680 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776f65d4_9fed_4694_8139_fb8bb9a95c7b.slice/crio-6f61f553f582f23a44d0e70aa1fe419b706c055c5e6ed0b8e84f86befaa3da38 WatchSource:0}: Error finding container 6f61f553f582f23a44d0e70aa1fe419b706c055c5e6ed0b8e84f86befaa3da38: Status 404 returned error can't find the container with id 6f61f553f582f23a44d0e70aa1fe419b706c055c5e6ed0b8e84f86befaa3da38 Sep 18 16:55:35.115683 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.115666 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 16:55:35.309157 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.309080 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9hg2q"] Sep 18 16:55:35.312725 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:55:35.312683 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2aace0d_8144_4803_b013_63eeaf617aca.slice/crio-0b362799f5ce4b2ad6da6c1ed2ed3c812626aa39851bdbdb98344f73bd849817 WatchSource:0}: Error finding container 0b362799f5ce4b2ad6da6c1ed2ed3c812626aa39851bdbdb98344f73bd849817: Status 404 returned error can't find the container with id 0b362799f5ce4b2ad6da6c1ed2ed3c812626aa39851bdbdb98344f73bd849817 Sep 18 16:55:35.313160 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:35.313135 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kld7t"] Sep 18 16:55:35.316261 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:55:35.316239 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab3a5c1_0c7d_40f8_b636_3e5bfe5b102f.slice/crio-5808cc9759643ddcff94a5510b7395183518ff7ccacd94cdd6bad0634a57e761 WatchSource:0}: Error finding container 5808cc9759643ddcff94a5510b7395183518ff7ccacd94cdd6bad0634a57e761: Status 404 returned error can't find the container with id 5808cc9759643ddcff94a5510b7395183518ff7ccacd94cdd6bad0634a57e761 Sep 18 16:55:36.091529 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:36.091442 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kld7t" event={"ID":"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f","Type":"ContainerStarted","Data":"5808cc9759643ddcff94a5510b7395183518ff7ccacd94cdd6bad0634a57e761"} Sep 18 16:55:36.093206 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:36.093175 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9hg2q" event={"ID":"b2aace0d-8144-4803-b013-63eeaf617aca","Type":"ContainerStarted","Data":"0b362799f5ce4b2ad6da6c1ed2ed3c812626aa39851bdbdb98344f73bd849817"} Sep 18 16:55:36.094873 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:36.094839 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5jqgt" event={"ID":"776f65d4-9fed-4694-8139-fb8bb9a95c7b","Type":"ContainerStarted","Data":"6f61f553f582f23a44d0e70aa1fe419b706c055c5e6ed0b8e84f86befaa3da38"} Sep 18 16:55:37.102095 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:37.102047 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5jqgt" event={"ID":"776f65d4-9fed-4694-8139-fb8bb9a95c7b","Type":"ContainerStarted","Data":"7171e46c5130afbdcd5cee700a029d574c917e0a5a15f7cda76004bb7691875b"} Sep 18 16:55:37.102581 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:37.102166 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5jqgt" event={"ID":"776f65d4-9fed-4694-8139-fb8bb9a95c7b","Type":"ContainerStarted","Data":"6d199d636f0f9109e0782b1abc3e255a2557300ae1a275bb61951beeb77afb9f"} Sep 18 16:55:37.124012 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:37.123946 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5jqgt" podStartSLOduration=376.974524925 podStartE2EDuration="6m18.123922616s" totalImagesPullingTime="1.149397691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:55:37.122666513 +0000 UTC m=+380.398791698" watchObservedRunningTime="2026-09-18 16:55:37.123922616 +0000 UTC m=+380.400047791" Sep 18 16:55:40.116071 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:40.116031 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kld7t" event={"ID":"2ab3a5c1-0c7d-40f8-b636-3e5bfe5b102f","Type":"ContainerStarted","Data":"9aa49d813a40017e3680e282cf7acb72b554011994d8afd018f352a34b89618b"} Sep 18 16:55:40.116510 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:40.116147 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:55:40.117571 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:40.117531 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9hg2q" event={"ID":"b2aace0d-8144-4803-b013-63eeaf617aca","Type":"ContainerStarted","Data":"7b342c9d63206c74942aca5172f5e9c31f0525ac680fa7e7f42914484f2c58c2"} Sep 18 16:55:40.132611 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:40.132529 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kld7t" podStartSLOduration=376.686857793 podStartE2EDuration="6m21.132511073s" totalImagesPullingTime="4.44565328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:55:40.132239124 +0000 UTC m=+383.408364319" watchObservedRunningTime="2026-09-18 16:55:40.132511073 +0000 UTC m=+383.408636246" Sep 18 16:55:40.147657 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:40.147599 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9hg2q" podStartSLOduration=376.694424867 podStartE2EDuration="6m21.147577535s" totalImagesPullingTime="4.453152668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:55:40.147224685 +0000 UTC m=+383.423349895" watchObservedRunningTime="2026-09-18 16:55:40.147577535 +0000 UTC m=+383.423702707" Sep 18 16:55:45.431134 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.431099 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b2m4k"] Sep 18 16:55:45.434258 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.434235 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:45.437827 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.437787 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bjwms\"" Sep 18 16:55:45.437984 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.437802 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 18 16:55:45.437984 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.437842 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 18 16:55:45.438704 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.438687 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 18 16:55:45.443019 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.442995 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b2m4k"] Sep 18 16:55:45.525578 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.525509 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljsz2\" (UniqueName: \"kubernetes.io/projected/066f325b-02fb-440a-bd43-8efdf244f004-kube-api-access-ljsz2\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:45.525578 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.525583 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066f325b-02fb-440a-bd43-8efdf244f004-cert\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:45.626903 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.626863 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066f325b-02fb-440a-bd43-8efdf244f004-cert\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:45.627074 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.626919 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljsz2\" (UniqueName: \"kubernetes.io/projected/066f325b-02fb-440a-bd43-8efdf244f004-kube-api-access-ljsz2\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:45.627074 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:55:45.627024 2586 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Sep 18 16:55:45.627156 ip-10-0-135-188 kubenswrapper[2586]: E0918 16:55:45.627092 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/066f325b-02fb-440a-bd43-8efdf244f004-cert podName:066f325b-02fb-440a-bd43-8efdf244f004 nodeName:}" failed. No retries permitted until 2026-09-18 16:55:46.127074757 +0000 UTC m=+389.403199910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/066f325b-02fb-440a-bd43-8efdf244f004-cert") pod "ingress-canary-b2m4k" (UID: "066f325b-02fb-440a-bd43-8efdf244f004") : secret "canary-serving-cert" not found Sep 18 16:55:45.653143 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:45.653105 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljsz2\" (UniqueName: \"kubernetes.io/projected/066f325b-02fb-440a-bd43-8efdf244f004-kube-api-access-ljsz2\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:46.131339 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:46.131239 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066f325b-02fb-440a-bd43-8efdf244f004-cert\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:46.133803 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:46.133777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066f325b-02fb-440a-bd43-8efdf244f004-cert\") pod \"ingress-canary-b2m4k\" (UID: \"066f325b-02fb-440a-bd43-8efdf244f004\") " pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:46.344452 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:46.344414 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b2m4k" Sep 18 16:55:46.622922 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:46.622884 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b2m4k"] Sep 18 16:55:46.626227 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:55:46.626202 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066f325b_02fb_440a_bd43_8efdf244f004.slice/crio-316b33491a186ab33acf5ad36f4771464798caffaed5c12b10ab80439d53e9d6 WatchSource:0}: Error finding container 316b33491a186ab33acf5ad36f4771464798caffaed5c12b10ab80439d53e9d6: Status 404 returned error can't find the container with id 316b33491a186ab33acf5ad36f4771464798caffaed5c12b10ab80439d53e9d6 Sep 18 16:55:47.146235 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:47.146183 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b2m4k" event={"ID":"066f325b-02fb-440a-bd43-8efdf244f004","Type":"ContainerStarted","Data":"316b33491a186ab33acf5ad36f4771464798caffaed5c12b10ab80439d53e9d6"} Sep 18 16:55:49.158492 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:49.158427 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b2m4k" event={"ID":"066f325b-02fb-440a-bd43-8efdf244f004","Type":"ContainerStarted","Data":"61f4852e693bbba5d4a6aa42122d9e36f3e7dbbfd1feb4960a66c960caf5050f"} Sep 18 16:55:49.179210 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:55:49.179155 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b2m4k" podStartSLOduration=2.172778731 podStartE2EDuration="4.17913712s" totalImagesPullingTime="2.006358389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:55:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:55:49.177409045 +0000 UTC m=+392.453534236" watchObservedRunningTime="2026-09-18 16:55:49.17913712 +0000 UTC m=+392.455262293" Sep 18 16:56:06.281210 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.281169 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-767f56548f-nr9g5"] Sep 18 16:56:06.286237 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.286208 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.288951 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.288925 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 18 16:56:06.290235 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290211 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5vmhh\"" Sep 18 16:56:06.290371 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290245 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 18 16:56:06.290371 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290291 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 18 16:56:06.290533 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290518 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 18 16:56:06.290620 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290592 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 18 16:56:06.290704 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290685 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 18 16:56:06.290772 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.290758 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 18 16:56:06.294946 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.294924 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 18 16:56:06.297898 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.297876 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767f56548f-nr9g5"] Sep 18 16:56:06.463840 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463800 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzlgl\" (UniqueName: \"kubernetes.io/projected/bde0e160-105b-411e-ba97-e90e27ce9570-kube-api-access-pzlgl\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.463840 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463842 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-oauth-config\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.464089 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463876 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-serving-cert\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.464089 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463898 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-console-config\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.464089 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463921 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-service-ca\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.464089 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463950 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-trusted-ca-bundle\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.464089 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.463969 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-oauth-serving-cert\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564377 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564276 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-service-ca\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564377 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564318 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-trusted-ca-bundle\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564377 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564336 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-oauth-serving-cert\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564690 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564384 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzlgl\" (UniqueName: \"kubernetes.io/projected/bde0e160-105b-411e-ba97-e90e27ce9570-kube-api-access-pzlgl\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564690 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564408 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-oauth-config\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564690 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564444 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-serving-cert\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.564690 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.564468 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-console-config\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.565181 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.565157 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-service-ca\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.565294 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.565273 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-console-config\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.565480 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.565457 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-oauth-serving-cert\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.565523 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.565492 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-trusted-ca-bundle\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.567073 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.567051 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-oauth-config\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.567230 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.567213 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-serving-cert\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.592009 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.591979 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzlgl\" (UniqueName: \"kubernetes.io/projected/bde0e160-105b-411e-ba97-e90e27ce9570-kube-api-access-pzlgl\") pod \"console-767f56548f-nr9g5\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.597868 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.597838 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:06.874510 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:06.874472 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767f56548f-nr9g5"] Sep 18 16:56:06.877676 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:56:06.877649 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde0e160_105b_411e_ba97_e90e27ce9570.slice/crio-53089694b5906651a11a60981083a4f056cdb683a1c821dd24fb7155e8f6c4cd WatchSource:0}: Error finding container 53089694b5906651a11a60981083a4f056cdb683a1c821dd24fb7155e8f6c4cd: Status 404 returned error can't find the container with id 53089694b5906651a11a60981083a4f056cdb683a1c821dd24fb7155e8f6c4cd Sep 18 16:56:07.219436 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:07.219347 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f56548f-nr9g5" event={"ID":"bde0e160-105b-411e-ba97-e90e27ce9570","Type":"ContainerStarted","Data":"53089694b5906651a11a60981083a4f056cdb683a1c821dd24fb7155e8f6c4cd"} Sep 18 16:56:11.123674 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:11.123635 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kld7t" Sep 18 16:56:11.235548 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:11.235518 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f56548f-nr9g5" event={"ID":"bde0e160-105b-411e-ba97-e90e27ce9570","Type":"ContainerStarted","Data":"3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e"} Sep 18 16:56:11.270938 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:11.270890 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-767f56548f-nr9g5" podStartSLOduration=1.451148978 podStartE2EDuration="5.270873841s" totalImagesPullingTime="3.819724863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:56:11.270085569 +0000 UTC m=+414.546210770" watchObservedRunningTime="2026-09-18 16:56:11.270873841 +0000 UTC m=+414.546999014" Sep 18 16:56:16.598071 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:16.598025 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:16.598469 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:16.598134 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:16.603000 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:16.602971 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:56:17.259809 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:56:17.259780 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 16:58:04.489052 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.488970 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz"] Sep 18 16:58:04.491763 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.491747 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.495040 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.495020 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 18 16:58:04.496354 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.496334 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 18 16:58:04.496418 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.496339 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 18 16:58:04.496418 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.496367 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 18 16:58:04.496509 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.496343 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-r4982\"" Sep 18 16:58:04.516653 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.516613 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz"] Sep 18 16:58:04.650401 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.650357 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d25e2f0-24b9-4b11-bff3-457b4f1b600e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6575d8f6fb-f44kz\" (UID: \"2d25e2f0-24b9-4b11-bff3-457b4f1b600e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.650619 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.650406 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdnd9\" (UniqueName: \"kubernetes.io/projected/2d25e2f0-24b9-4b11-bff3-457b4f1b600e-kube-api-access-wdnd9\") pod \"managed-serviceaccount-addon-agent-6575d8f6fb-f44kz\" (UID: \"2d25e2f0-24b9-4b11-bff3-457b4f1b600e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.750899 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.750812 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d25e2f0-24b9-4b11-bff3-457b4f1b600e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6575d8f6fb-f44kz\" (UID: \"2d25e2f0-24b9-4b11-bff3-457b4f1b600e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.750899 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.750855 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdnd9\" (UniqueName: \"kubernetes.io/projected/2d25e2f0-24b9-4b11-bff3-457b4f1b600e-kube-api-access-wdnd9\") pod \"managed-serviceaccount-addon-agent-6575d8f6fb-f44kz\" (UID: \"2d25e2f0-24b9-4b11-bff3-457b4f1b600e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.753720 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.753696 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d25e2f0-24b9-4b11-bff3-457b4f1b600e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6575d8f6fb-f44kz\" (UID: \"2d25e2f0-24b9-4b11-bff3-457b4f1b600e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.754280 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.754263 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts"] Sep 18 16:58:04.757249 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.757227 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.760765 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.760742 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 18 16:58:04.760879 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.760750 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 18 16:58:04.761085 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.761070 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 18 16:58:04.761691 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.761676 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 18 16:58:04.762319 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.762299 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdnd9\" (UniqueName: \"kubernetes.io/projected/2d25e2f0-24b9-4b11-bff3-457b4f1b600e-kube-api-access-wdnd9\") pod \"managed-serviceaccount-addon-agent-6575d8f6fb-f44kz\" (UID: \"2d25e2f0-24b9-4b11-bff3-457b4f1b600e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.772464 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.772437 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts"] Sep 18 16:58:04.809819 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.809773 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" Sep 18 16:58:04.851900 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.851865 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-ca\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.851900 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.851905 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75rh8\" (UniqueName: \"kubernetes.io/projected/823ae945-afe1-4c74-8664-2729ad5a9b22-kube-api-access-75rh8\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.852130 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.851930 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-hub\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.852130 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.851954 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ae945-afe1-4c74-8664-2729ad5a9b22-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.852130 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.852001 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.852130 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.852029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.953116 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.953080 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-ca\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.953116 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.953128 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75rh8\" (UniqueName: \"kubernetes.io/projected/823ae945-afe1-4c74-8664-2729ad5a9b22-kube-api-access-75rh8\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.953387 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.953170 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-hub\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.953387 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.953196 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ae945-afe1-4c74-8664-2729ad5a9b22-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.953387 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.953223 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.953387 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.953258 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.954199 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.954171 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/823ae945-afe1-4c74-8664-2729ad5a9b22-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.955849 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.955822 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-ca\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.955956 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.955901 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.955956 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.955920 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-hub\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.956052 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.956029 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/823ae945-afe1-4c74-8664-2729ad5a9b22-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:04.963460 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:04.963439 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75rh8\" (UniqueName: \"kubernetes.io/projected/823ae945-afe1-4c74-8664-2729ad5a9b22-kube-api-access-75rh8\") pod \"cluster-proxy-proxy-agent-7cfdc77c6c-pnxts\" (UID: \"823ae945-afe1-4c74-8664-2729ad5a9b22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:05.071351 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:05.071263 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" Sep 18 16:58:05.081596 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:05.081499 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz"] Sep 18 16:58:05.086402 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:58:05.086376 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d25e2f0_24b9_4b11_bff3_457b4f1b600e.slice/crio-082b529f88a9ffbfe9deb308911d1fcef6ccbedf1036db7b1bd9db9f80bd3114 WatchSource:0}: Error finding container 082b529f88a9ffbfe9deb308911d1fcef6ccbedf1036db7b1bd9db9f80bd3114: Status 404 returned error can't find the container with id 082b529f88a9ffbfe9deb308911d1fcef6ccbedf1036db7b1bd9db9f80bd3114 Sep 18 16:58:05.350671 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:05.350638 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts"] Sep 18 16:58:05.354948 ip-10-0-135-188 kubenswrapper[2586]: W0918 16:58:05.354921 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod823ae945_afe1_4c74_8664_2729ad5a9b22.slice/crio-b1bd516d88b0aef9c4216146ce79849689a0eaea1aec59d96671d3c783047127 WatchSource:0}: Error finding container b1bd516d88b0aef9c4216146ce79849689a0eaea1aec59d96671d3c783047127: Status 404 returned error can't find the container with id b1bd516d88b0aef9c4216146ce79849689a0eaea1aec59d96671d3c783047127 Sep 18 16:58:05.602712 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:05.602612 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" event={"ID":"2d25e2f0-24b9-4b11-bff3-457b4f1b600e","Type":"ContainerStarted","Data":"082b529f88a9ffbfe9deb308911d1fcef6ccbedf1036db7b1bd9db9f80bd3114"} Sep 18 16:58:05.604404 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:05.604369 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" event={"ID":"823ae945-afe1-4c74-8664-2729ad5a9b22","Type":"ContainerStarted","Data":"b1bd516d88b0aef9c4216146ce79849689a0eaea1aec59d96671d3c783047127"} Sep 18 16:58:07.613363 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:07.613331 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" event={"ID":"2d25e2f0-24b9-4b11-bff3-457b4f1b600e","Type":"ContainerStarted","Data":"8b89f42dc1a7aa331e6443fbde991068de8358e520d68c7a72407925e100846f"} Sep 18 16:58:07.632354 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:07.632285 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6575d8f6fb-f44kz" podStartSLOduration=1.220096748 podStartE2EDuration="3.632261793s" totalImagesPullingTime="2.412165045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:58:07.629935797 +0000 UTC m=+530.906060976" watchObservedRunningTime="2026-09-18 16:58:07.632261793 +0000 UTC m=+530.908386967" Sep 18 16:58:10.626695 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:10.626659 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" event={"ID":"823ae945-afe1-4c74-8664-2729ad5a9b22","Type":"ContainerStarted","Data":"949cc8297b10baa8f7947fbef3a48fdc91735cfdb1fe757658ae0774d33c3235"} Sep 18 16:58:10.626695 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:10.626704 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" event={"ID":"823ae945-afe1-4c74-8664-2729ad5a9b22","Type":"ContainerStarted","Data":"786bbe400807bae69dfe4976d04d5f135b128b39159f92a83fb6d18bb3172310"} Sep 18 16:58:10.627121 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:10.626717 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" event={"ID":"823ae945-afe1-4c74-8664-2729ad5a9b22","Type":"ContainerStarted","Data":"7556708daba1b16aceff97928972e24872d6e6b2c7e71830cbcd3655667f617d"} Sep 18 16:58:10.645690 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:58:10.645574 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cfdc77c6c-pnxts" podStartSLOduration=1.621496738 podStartE2EDuration="6.645532371s" totalImagesPullingTime="5.024035633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 16:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 16:58:10.644999216 +0000 UTC m=+533.921124391" watchObservedRunningTime="2026-09-18 16:58:10.645532371 +0000 UTC m=+533.921657543" Sep 18 16:59:18.815721 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:59:18.815695 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:59:18.817761 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:59:18.817740 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 16:59:18.840863 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:59:18.840836 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 16:59:18.842723 ip-10-0-135-188 kubenswrapper[2586]: I0918 16:59:18.842695 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:00:13.336502 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.336458 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-595f6fb4d6-csqxm"] Sep 18 17:00:13.339631 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.339606 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.363595 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.363549 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595f6fb4d6-csqxm"] Sep 18 17:00:13.428187 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428144 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-service-ca\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.428187 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428186 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-trusted-ca-bundle\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.428405 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428214 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-serving-cert\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.428405 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428229 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-config\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.428405 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428267 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpzzb\" (UniqueName: \"kubernetes.io/projected/f1bee90d-f68c-459c-886d-48fb82ba3c48-kube-api-access-mpzzb\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.428405 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428376 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-oauth-config\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.428529 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.428415 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-oauth-serving-cert\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.529686 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529653 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-serving-cert\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.529686 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529686 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-config\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.529846 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529706 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpzzb\" (UniqueName: \"kubernetes.io/projected/f1bee90d-f68c-459c-886d-48fb82ba3c48-kube-api-access-mpzzb\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.529846 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529733 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-oauth-config\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.529846 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529766 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-oauth-serving-cert\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.529846 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529826 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-service-ca\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.530106 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.529850 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-trusted-ca-bundle\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.530510 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.530486 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-config\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.530618 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.530538 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-service-ca\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.530713 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.530688 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-oauth-serving-cert\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.530864 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.530848 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-trusted-ca-bundle\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.532882 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.532858 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-oauth-config\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.533078 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.533056 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-serving-cert\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.539923 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.539891 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpzzb\" (UniqueName: \"kubernetes.io/projected/f1bee90d-f68c-459c-886d-48fb82ba3c48-kube-api-access-mpzzb\") pod \"console-595f6fb4d6-csqxm\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.648694 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.648606 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:13.924437 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:13.924351 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595f6fb4d6-csqxm"] Sep 18 17:00:13.927705 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:00:13.927675 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1bee90d_f68c_459c_886d_48fb82ba3c48.slice/crio-3c418432e5ebca97aa5c7627e581f78fe3c86721fc2cc09ff043c511ece9f992 WatchSource:0}: Error finding container 3c418432e5ebca97aa5c7627e581f78fe3c86721fc2cc09ff043c511ece9f992: Status 404 returned error can't find the container with id 3c418432e5ebca97aa5c7627e581f78fe3c86721fc2cc09ff043c511ece9f992 Sep 18 17:00:14.035410 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:14.035372 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595f6fb4d6-csqxm" event={"ID":"f1bee90d-f68c-459c-886d-48fb82ba3c48","Type":"ContainerStarted","Data":"58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904"} Sep 18 17:00:14.035632 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:14.035429 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595f6fb4d6-csqxm" event={"ID":"f1bee90d-f68c-459c-886d-48fb82ba3c48","Type":"ContainerStarted","Data":"3c418432e5ebca97aa5c7627e581f78fe3c86721fc2cc09ff043c511ece9f992"} Sep 18 17:00:14.056846 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:14.056796 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-595f6fb4d6-csqxm" podStartSLOduration=1.056778633 podStartE2EDuration="1.056778633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:00:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:00:14.056051569 +0000 UTC m=+657.332176772" watchObservedRunningTime="2026-09-18 17:00:14.056778633 +0000 UTC m=+657.332903806" Sep 18 17:00:23.648925 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:23.648888 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:23.649414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:23.648950 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:23.653893 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:23.653867 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:24.074287 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:24.074197 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:00:24.124209 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:24.124168 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767f56548f-nr9g5"] Sep 18 17:00:49.144296 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.144247 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-767f56548f-nr9g5" podUID="bde0e160-105b-411e-ba97-e90e27ce9570" containerName="console" containerID="cri-o://3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e" gracePeriod=15 Sep 18 17:00:49.386645 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.386612 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 17:00:49.480616 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480495 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-oauth-serving-cert\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.480616 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480535 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-service-ca\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.480616 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480580 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-serving-cert\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.480616 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480604 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-console-config\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.480946 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480624 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-trusted-ca-bundle\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.480946 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480644 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzlgl\" (UniqueName: \"kubernetes.io/projected/bde0e160-105b-411e-ba97-e90e27ce9570-kube-api-access-pzlgl\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.480946 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.480673 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-oauth-config\") pod \"bde0e160-105b-411e-ba97-e90e27ce9570\" (UID: \"bde0e160-105b-411e-ba97-e90e27ce9570\") " Sep 18 17:00:49.481091 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.481044 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-service-ca" (OuterVolumeSpecName: "service-ca") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:00:49.481091 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.481054 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-console-config" (OuterVolumeSpecName: "console-config") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:00:49.481091 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.481058 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:00:49.481217 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.481135 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:00:49.483286 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.483241 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:00:49.483286 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.483271 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:00:49.483445 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.483319 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bde0e160-105b-411e-ba97-e90e27ce9570-kube-api-access-pzlgl" (OuterVolumeSpecName: "kube-api-access-pzlgl") pod "bde0e160-105b-411e-ba97-e90e27ce9570" (UID: "bde0e160-105b-411e-ba97-e90e27ce9570"). InnerVolumeSpecName "kube-api-access-pzlgl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:00:49.581865 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581818 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-service-ca\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:49.581865 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581860 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-serving-cert\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:49.581865 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581869 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-console-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:49.581865 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581879 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-trusted-ca-bundle\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:49.582096 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581888 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzlgl\" (UniqueName: \"kubernetes.io/projected/bde0e160-105b-411e-ba97-e90e27ce9570-kube-api-access-pzlgl\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:49.582096 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581896 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bde0e160-105b-411e-ba97-e90e27ce9570-console-oauth-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:49.582096 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:49.581905 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bde0e160-105b-411e-ba97-e90e27ce9570-oauth-serving-cert\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:00:50.161626 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.161591 2586 generic.go:356] "Generic (PLEG): container finished" podID="bde0e160-105b-411e-ba97-e90e27ce9570" containerID="3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e" exitCode=0 Sep 18 17:00:50.162066 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.161697 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f56548f-nr9g5" event={"ID":"bde0e160-105b-411e-ba97-e90e27ce9570","Type":"ContainerDied","Data":"3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e"} Sep 18 17:00:50.162066 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.161713 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767f56548f-nr9g5" Sep 18 17:00:50.162066 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.161750 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767f56548f-nr9g5" event={"ID":"bde0e160-105b-411e-ba97-e90e27ce9570","Type":"ContainerDied","Data":"53089694b5906651a11a60981083a4f056cdb683a1c821dd24fb7155e8f6c4cd"} Sep 18 17:00:50.162066 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.161778 2586 scope.go:117] "RemoveContainer" containerID="3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e" Sep 18 17:00:50.171135 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.171108 2586 scope.go:117] "RemoveContainer" containerID="3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e" Sep 18 17:00:50.171487 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:00:50.171463 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e\": container with ID starting with 3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e not found: ID does not exist" containerID="3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e" Sep 18 17:00:50.171595 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.171497 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e"} err="failed to get container status \"3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e\": rpc error: code = NotFound desc = could not find container \"3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e\": container with ID starting with 3c2e5064a31fc30e0c1015846310ca723913557f5044170fcd3adc9f1d47c22e not found: ID does not exist" Sep 18 17:00:50.184142 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.184102 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767f56548f-nr9g5"] Sep 18 17:00:50.188709 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.188680 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-767f56548f-nr9g5"] Sep 18 17:00:50.914936 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:50.914895 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bde0e160-105b-411e-ba97-e90e27ce9570" path="/var/lib/kubelet/pods/bde0e160-105b-411e-ba97-e90e27ce9570/volumes" Sep 18 17:00:54.867165 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.867132 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b4fc4cb9c-fwr7b"] Sep 18 17:00:54.867681 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.867352 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bde0e160-105b-411e-ba97-e90e27ce9570" containerName="console" Sep 18 17:00:54.867681 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.867362 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bde0e160-105b-411e-ba97-e90e27ce9570" containerName="console" Sep 18 17:00:54.867681 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.867404 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bde0e160-105b-411e-ba97-e90e27ce9570" containerName="console" Sep 18 17:00:54.871704 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.871681 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.887055 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.887021 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4fc4cb9c-fwr7b"] Sep 18 17:00:54.920591 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920535 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-oauth-serving-cert\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.920759 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920596 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-config\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.920759 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920650 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-service-ca\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.920759 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920718 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-oauth-config\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.920759 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920744 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-trusted-ca-bundle\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.920891 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920773 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-serving-cert\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:54.920891 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:54.920797 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7tr6\" (UniqueName: \"kubernetes.io/projected/27c8f7d0-221c-4284-a9b7-fe602797e70d-kube-api-access-v7tr6\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022134 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022097 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-config\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022134 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022135 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-service-ca\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022285 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022161 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-oauth-config\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022320 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022291 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-trusted-ca-bundle\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022353 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022322 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-serving-cert\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022353 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022345 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7tr6\" (UniqueName: \"kubernetes.io/projected/27c8f7d0-221c-4284-a9b7-fe602797e70d-kube-api-access-v7tr6\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.022440 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.022381 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-oauth-serving-cert\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.023076 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.023051 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-oauth-serving-cert\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.023193 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.023053 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-service-ca\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.023193 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.023053 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-config\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.023483 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.023458 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c8f7d0-221c-4284-a9b7-fe602797e70d-trusted-ca-bundle\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.024837 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.024819 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-oauth-config\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.024955 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.024930 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c8f7d0-221c-4284-a9b7-fe602797e70d-console-serving-cert\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.035386 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.035351 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7tr6\" (UniqueName: \"kubernetes.io/projected/27c8f7d0-221c-4284-a9b7-fe602797e70d-kube-api-access-v7tr6\") pod \"console-7b4fc4cb9c-fwr7b\" (UID: \"27c8f7d0-221c-4284-a9b7-fe602797e70d\") " pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.181452 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.181370 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:00:55.459922 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:55.459894 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b4fc4cb9c-fwr7b"] Sep 18 17:00:55.462342 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:00:55.462314 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c8f7d0_221c_4284_a9b7_fe602797e70d.slice/crio-c4b992334e710193159fa8e5af60cdd1270a7f1289a7c14d7accff4f2bfe0abe WatchSource:0}: Error finding container c4b992334e710193159fa8e5af60cdd1270a7f1289a7c14d7accff4f2bfe0abe: Status 404 returned error can't find the container with id c4b992334e710193159fa8e5af60cdd1270a7f1289a7c14d7accff4f2bfe0abe Sep 18 17:00:56.183917 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:56.183881 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4fc4cb9c-fwr7b" event={"ID":"27c8f7d0-221c-4284-a9b7-fe602797e70d","Type":"ContainerStarted","Data":"6354532dd5ae4e5fe6a21d731d8045fbb5bce87dc5f7c437482968628eaae6ca"} Sep 18 17:00:56.184309 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:56.183960 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b4fc4cb9c-fwr7b" event={"ID":"27c8f7d0-221c-4284-a9b7-fe602797e70d","Type":"ContainerStarted","Data":"c4b992334e710193159fa8e5af60cdd1270a7f1289a7c14d7accff4f2bfe0abe"} Sep 18 17:00:56.203659 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:00:56.203604 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b4fc4cb9c-fwr7b" podStartSLOduration=2.203586506 podStartE2EDuration="2.203586506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:00:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:00:56.202339328 +0000 UTC m=+699.478464521" watchObservedRunningTime="2026-09-18 17:00:56.203586506 +0000 UTC m=+699.479711679" Sep 18 17:01:05.182395 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:05.182293 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:01:05.182395 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:05.182371 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:01:05.187487 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:05.187453 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:01:05.220674 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:05.220644 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b4fc4cb9c-fwr7b" Sep 18 17:01:05.279270 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:05.279230 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-595f6fb4d6-csqxm"] Sep 18 17:01:28.351736 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.351701 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb"] Sep 18 17:01:28.355740 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.355720 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.358404 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.358374 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 18 17:01:28.358537 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.358408 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 18 17:01:28.359524 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.359506 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2h7fc\"" Sep 18 17:01:28.363912 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.363886 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb"] Sep 18 17:01:28.461523 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.461487 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.461723 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.461549 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwskz\" (UniqueName: \"kubernetes.io/projected/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-kube-api-access-gwskz\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.461723 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.461676 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.562956 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.562915 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwskz\" (UniqueName: \"kubernetes.io/projected/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-kube-api-access-gwskz\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.563120 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.562970 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.563120 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.562993 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.563320 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.563305 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.563374 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.563355 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.576334 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.576301 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwskz\" (UniqueName: \"kubernetes.io/projected/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-kube-api-access-gwskz\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.665620 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.665498 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:28.941865 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.941773 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb"] Sep 18 17:01:28.944845 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:01:28.944817 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05ba3b0_b6b6_40b0_960d_e3b3243c6c9a.slice/crio-aabc6b3e2095c73f3cc8fe99124d60f9aceb6cf8054eacb9e01d45700c5a6c0d WatchSource:0}: Error finding container aabc6b3e2095c73f3cc8fe99124d60f9aceb6cf8054eacb9e01d45700c5a6c0d: Status 404 returned error can't find the container with id aabc6b3e2095c73f3cc8fe99124d60f9aceb6cf8054eacb9e01d45700c5a6c0d Sep 18 17:01:28.946969 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:28.946949 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 17:01:29.299508 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:29.299419 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" event={"ID":"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a","Type":"ContainerStarted","Data":"aabc6b3e2095c73f3cc8fe99124d60f9aceb6cf8054eacb9e01d45700c5a6c0d"} Sep 18 17:01:30.299166 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.299082 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-595f6fb4d6-csqxm" podUID="f1bee90d-f68c-459c-886d-48fb82ba3c48" containerName="console" containerID="cri-o://58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904" gracePeriod=15 Sep 18 17:01:30.568942 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.568910 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:01:30.681603 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681537 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-oauth-config\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.681603 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681612 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-oauth-serving-cert\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.681866 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681644 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-config\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.681866 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681665 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-trusted-ca-bundle\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.681866 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681687 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpzzb\" (UniqueName: \"kubernetes.io/projected/f1bee90d-f68c-459c-886d-48fb82ba3c48-kube-api-access-mpzzb\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.681866 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681712 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-service-ca\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.681866 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.681735 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-serving-cert\") pod \"f1bee90d-f68c-459c-886d-48fb82ba3c48\" (UID: \"f1bee90d-f68c-459c-886d-48fb82ba3c48\") " Sep 18 17:01:30.682167 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.682115 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:01:30.682167 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.682126 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-config" (OuterVolumeSpecName: "console-config") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:01:30.682167 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.682144 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:01:30.682348 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.682233 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-service-ca" (OuterVolumeSpecName: "service-ca") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:01:30.683983 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.683951 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:01:30.684097 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.683990 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1bee90d-f68c-459c-886d-48fb82ba3c48-kube-api-access-mpzzb" (OuterVolumeSpecName: "kube-api-access-mpzzb") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "kube-api-access-mpzzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:01:30.684097 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.684027 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f1bee90d-f68c-459c-886d-48fb82ba3c48" (UID: "f1bee90d-f68c-459c-886d-48fb82ba3c48"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:01:30.782835 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782791 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:30.782835 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782833 2586 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-trusted-ca-bundle\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:30.783080 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782850 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpzzb\" (UniqueName: \"kubernetes.io/projected/f1bee90d-f68c-459c-886d-48fb82ba3c48-kube-api-access-mpzzb\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:30.783080 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782864 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-service-ca\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:30.783080 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782878 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-serving-cert\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:30.783080 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782889 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f1bee90d-f68c-459c-886d-48fb82ba3c48-console-oauth-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:30.783080 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:30.782903 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f1bee90d-f68c-459c-886d-48fb82ba3c48-oauth-serving-cert\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:31.309512 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.309416 2586 generic.go:356] "Generic (PLEG): container finished" podID="f1bee90d-f68c-459c-886d-48fb82ba3c48" containerID="58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904" exitCode=0 Sep 18 17:01:31.309512 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.309476 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595f6fb4d6-csqxm" Sep 18 17:01:31.309512 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.309497 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595f6fb4d6-csqxm" event={"ID":"f1bee90d-f68c-459c-886d-48fb82ba3c48","Type":"ContainerDied","Data":"58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904"} Sep 18 17:01:31.310096 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.309582 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595f6fb4d6-csqxm" event={"ID":"f1bee90d-f68c-459c-886d-48fb82ba3c48","Type":"ContainerDied","Data":"3c418432e5ebca97aa5c7627e581f78fe3c86721fc2cc09ff043c511ece9f992"} Sep 18 17:01:31.310096 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.309615 2586 scope.go:117] "RemoveContainer" containerID="58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904" Sep 18 17:01:31.319929 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.319904 2586 scope.go:117] "RemoveContainer" containerID="58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904" Sep 18 17:01:31.320361 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:01:31.320320 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904\": container with ID starting with 58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904 not found: ID does not exist" containerID="58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904" Sep 18 17:01:31.320477 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.320358 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904"} err="failed to get container status \"58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904\": rpc error: code = NotFound desc = could not find container \"58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904\": container with ID starting with 58807f847f6e7aee0fd975e9a7f723c6e6568f4a55b0f6f7b54041baaf699904 not found: ID does not exist" Sep 18 17:01:31.330418 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.330380 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-595f6fb4d6-csqxm"] Sep 18 17:01:31.334909 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:31.334874 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-595f6fb4d6-csqxm"] Sep 18 17:01:32.917148 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:32.917112 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1bee90d-f68c-459c-886d-48fb82ba3c48" path="/var/lib/kubelet/pods/f1bee90d-f68c-459c-886d-48fb82ba3c48/volumes" Sep 18 17:01:36.332241 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:36.332207 2586 generic.go:356] "Generic (PLEG): container finished" podID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerID="b0c393a7871688e78a5fe7399743d1e698818c5f99d4c3e7336899740ba45e85" exitCode=0 Sep 18 17:01:36.332685 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:36.332271 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" event={"ID":"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a","Type":"ContainerDied","Data":"b0c393a7871688e78a5fe7399743d1e698818c5f99d4c3e7336899740ba45e85"} Sep 18 17:01:48.378022 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:48.377980 2586 generic.go:356] "Generic (PLEG): container finished" podID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerID="e056db07e0a1bb053e2f4dfd3417907cde6364b1ba108a7a8a6754bf9ed8d0ed" exitCode=0 Sep 18 17:01:48.378434 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:48.378045 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" event={"ID":"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a","Type":"ContainerDied","Data":"e056db07e0a1bb053e2f4dfd3417907cde6364b1ba108a7a8a6754bf9ed8d0ed"} Sep 18 17:01:55.408303 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:55.408266 2586 generic.go:356] "Generic (PLEG): container finished" podID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerID="289870a02afd6338d94d290175b64f88951cef1fc4fb665dfc27e75f0eb427ad" exitCode=0 Sep 18 17:01:55.408772 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:55.408361 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" event={"ID":"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a","Type":"ContainerDied","Data":"289870a02afd6338d94d290175b64f88951cef1fc4fb665dfc27e75f0eb427ad"} Sep 18 17:01:56.539766 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.539731 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:56.689636 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.689505 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-util\") pod \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " Sep 18 17:01:56.689636 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.689581 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-bundle\") pod \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " Sep 18 17:01:56.689636 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.689638 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwskz\" (UniqueName: \"kubernetes.io/projected/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-kube-api-access-gwskz\") pod \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\" (UID: \"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a\") " Sep 18 17:01:56.690295 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.690256 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-bundle" (OuterVolumeSpecName: "bundle") pod "c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" (UID: "c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 17:01:56.692048 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.692019 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-kube-api-access-gwskz" (OuterVolumeSpecName: "kube-api-access-gwskz") pod "c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" (UID: "c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a"). InnerVolumeSpecName "kube-api-access-gwskz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:01:56.694656 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.694630 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-util" (OuterVolumeSpecName: "util") pod "c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" (UID: "c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 18 17:01:56.790400 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.790369 2586 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-util\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:56.790400 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.790397 2586 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-bundle\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:56.790400 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:56.790407 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwskz\" (UniqueName: \"kubernetes.io/projected/c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a-kube-api-access-gwskz\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:01:57.416994 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:57.416960 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" event={"ID":"c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a","Type":"ContainerDied","Data":"aabc6b3e2095c73f3cc8fe99124d60f9aceb6cf8054eacb9e01d45700c5a6c0d"} Sep 18 17:01:57.416994 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:57.416996 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1a4z6cb" Sep 18 17:01:57.417208 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:01:57.417005 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aabc6b3e2095c73f3cc8fe99124d60f9aceb6cf8054eacb9e01d45700c5a6c0d" Sep 18 17:02:01.580494 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580463 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz"] Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580805 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1bee90d-f68c-459c-886d-48fb82ba3c48" containerName="console" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580825 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1bee90d-f68c-459c-886d-48fb82ba3c48" containerName="console" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580836 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="extract" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580845 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="extract" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580857 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="pull" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580865 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="pull" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580875 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="util" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580882 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="util" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580942 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c05ba3b0-b6b6-40b0-960d-e3b3243c6c9a" containerName="extract" Sep 18 17:02:01.581025 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.580956 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1bee90d-f68c-459c-886d-48fb82ba3c48" containerName="console" Sep 18 17:02:01.583080 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.583058 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.585713 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.585686 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Sep 18 17:02:01.585959 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.585939 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Sep 18 17:02:01.586054 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.585973 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Sep 18 17:02:01.586054 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.585983 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-rw6pg\"" Sep 18 17:02:01.595929 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.595851 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz"] Sep 18 17:02:01.725020 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.724987 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k89mj\" (UniqueName: \"kubernetes.io/projected/32d861db-b3f9-4e53-a0ec-8119a4487c6b-kube-api-access-k89mj\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cp5fz\" (UID: \"32d861db-b3f9-4e53-a0ec-8119a4487c6b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.725189 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.725029 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/32d861db-b3f9-4e53-a0ec-8119a4487c6b-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cp5fz\" (UID: \"32d861db-b3f9-4e53-a0ec-8119a4487c6b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.826039 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.825995 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k89mj\" (UniqueName: \"kubernetes.io/projected/32d861db-b3f9-4e53-a0ec-8119a4487c6b-kube-api-access-k89mj\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cp5fz\" (UID: \"32d861db-b3f9-4e53-a0ec-8119a4487c6b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.826218 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.826053 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/32d861db-b3f9-4e53-a0ec-8119a4487c6b-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cp5fz\" (UID: \"32d861db-b3f9-4e53-a0ec-8119a4487c6b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.828650 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.828623 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/32d861db-b3f9-4e53-a0ec-8119a4487c6b-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cp5fz\" (UID: \"32d861db-b3f9-4e53-a0ec-8119a4487c6b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.838627 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.838529 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k89mj\" (UniqueName: \"kubernetes.io/projected/32d861db-b3f9-4e53-a0ec-8119a4487c6b-kube-api-access-k89mj\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-cp5fz\" (UID: \"32d861db-b3f9-4e53-a0ec-8119a4487c6b\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:01.893138 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:01.893100 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:02.176193 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:02.176095 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz"] Sep 18 17:02:02.179295 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:02:02.179263 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d861db_b3f9_4e53_a0ec_8119a4487c6b.slice/crio-9ca82ed47e7f6d2677d3681bf5e0c4b9cf897b564dcb7664d19dea680a7035a5 WatchSource:0}: Error finding container 9ca82ed47e7f6d2677d3681bf5e0c4b9cf897b564dcb7664d19dea680a7035a5: Status 404 returned error can't find the container with id 9ca82ed47e7f6d2677d3681bf5e0c4b9cf897b564dcb7664d19dea680a7035a5 Sep 18 17:02:02.436490 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:02.436406 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" event={"ID":"32d861db-b3f9-4e53-a0ec-8119a4487c6b","Type":"ContainerStarted","Data":"9ca82ed47e7f6d2677d3681bf5e0c4b9cf897b564dcb7664d19dea680a7035a5"} Sep 18 17:02:06.457741 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.457702 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" event={"ID":"32d861db-b3f9-4e53-a0ec-8119a4487c6b","Type":"ContainerStarted","Data":"d97b5fe4237c984773516a3cddd3a9aeb249196808a45eeea1bb328a1a44c00a"} Sep 18 17:02:06.458154 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.457839 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:06.481417 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.481354 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" podStartSLOduration=1.9116549840000001 podStartE2EDuration="5.481328237s" totalImagesPullingTime="3.569673253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:02:06.479569134 +0000 UTC m=+769.755694317" watchObservedRunningTime="2026-09-18 17:02:06.481328237 +0000 UTC m=+769.757453412" Sep 18 17:02:06.503816 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.503780 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-mw86p"] Sep 18 17:02:06.506615 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.506583 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.510722 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.510698 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-wz2fw\"" Sep 18 17:02:06.510873 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.510739 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Sep 18 17:02:06.510873 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.510740 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Sep 18 17:02:06.518768 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.518739 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-mw86p"] Sep 18 17:02:06.662767 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.662729 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-cabundle0\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.662930 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.662777 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.662930 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.662810 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrjdm\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-kube-api-access-rrjdm\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.763851 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.763744 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-cabundle0\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.763851 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.763798 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.763851 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.763824 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrjdm\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-kube-api-access-rrjdm\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.764145 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:06.763973 2586 secret.go:281] references non-existent secret key: ca.crt Sep 18 17:02:06.764145 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:06.764001 2586 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Sep 18 17:02:06.764145 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:06.764012 2586 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-mw86p: references non-existent secret key: ca.crt Sep 18 17:02:06.764145 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:06.764077 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates podName:7e70dd34-5b2c-4b87-bdb9-672855bf4bf4 nodeName:}" failed. No retries permitted until 2026-09-18 17:02:07.264057367 +0000 UTC m=+770.540182522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates") pod "keda-operator-579c8958f7-mw86p" (UID: "7e70dd34-5b2c-4b87-bdb9-672855bf4bf4") : references non-existent secret key: ca.crt Sep 18 17:02:06.764583 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.764541 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-cabundle0\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:06.780845 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:06.780817 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrjdm\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-kube-api-access-rrjdm\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:07.003612 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.003548 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj"] Sep 18 17:02:07.005795 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.005778 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.008703 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.008681 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Sep 18 17:02:07.023022 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.022938 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj"] Sep 18 17:02:07.065733 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.065699 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-certificates\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.065925 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.065829 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.065925 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.065872 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbqvw\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-kube-api-access-mbqvw\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.166725 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.166691 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-certificates\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.166927 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.166793 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.166927 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.166819 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbqvw\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-kube-api-access-mbqvw\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.166927 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.166852 2586 secret.go:281] references non-existent secret key: tls.crt Sep 18 17:02:07.166927 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.166879 2586 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Sep 18 17:02:07.166927 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.166902 2586 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj: references non-existent secret key: tls.crt Sep 18 17:02:07.167185 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.166974 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-certificates podName:6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4 nodeName:}" failed. No retries permitted until 2026-09-18 17:02:07.666951394 +0000 UTC m=+770.943076546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-certificates") pod "keda-metrics-apiserver-77857b7b98-d4xmj" (UID: "6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4") : references non-existent secret key: tls.crt Sep 18 17:02:07.167234 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.167188 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.186786 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.186750 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbqvw\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-kube-api-access-mbqvw\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.268006 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.267969 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:07.268175 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.268095 2586 secret.go:281] references non-existent secret key: ca.crt Sep 18 17:02:07.268175 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.268109 2586 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Sep 18 17:02:07.268175 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.268117 2586 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-mw86p: references non-existent secret key: ca.crt Sep 18 17:02:07.268175 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:02:07.268171 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates podName:7e70dd34-5b2c-4b87-bdb9-672855bf4bf4 nodeName:}" failed. No retries permitted until 2026-09-18 17:02:08.268155351 +0000 UTC m=+771.544280506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates") pod "keda-operator-579c8958f7-mw86p" (UID: "7e70dd34-5b2c-4b87-bdb9-672855bf4bf4") : references non-existent secret key: ca.crt Sep 18 17:02:07.524541 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.524501 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-m5hrw"] Sep 18 17:02:07.526881 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.526856 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.529752 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.529733 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Sep 18 17:02:07.539716 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.539667 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-m5hrw"] Sep 18 17:02:07.570134 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.570087 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2f2baaa4-05b4-4efc-bc4e-476d4314d0e5-certificates\") pod \"keda-admission-6c4d864b86-m5hrw\" (UID: \"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5\") " pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.570310 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.570200 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lkpb\" (UniqueName: \"kubernetes.io/projected/2f2baaa4-05b4-4efc-bc4e-476d4314d0e5-kube-api-access-8lkpb\") pod \"keda-admission-6c4d864b86-m5hrw\" (UID: \"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5\") " pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.671126 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.671084 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lkpb\" (UniqueName: \"kubernetes.io/projected/2f2baaa4-05b4-4efc-bc4e-476d4314d0e5-kube-api-access-8lkpb\") pod \"keda-admission-6c4d864b86-m5hrw\" (UID: \"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5\") " pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.671277 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.671139 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2f2baaa4-05b4-4efc-bc4e-476d4314d0e5-certificates\") pod \"keda-admission-6c4d864b86-m5hrw\" (UID: \"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5\") " pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.671277 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.671256 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-certificates\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.673955 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.673930 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2f2baaa4-05b4-4efc-bc4e-476d4314d0e5-certificates\") pod \"keda-admission-6c4d864b86-m5hrw\" (UID: \"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5\") " pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.674058 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.674005 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4-certificates\") pod \"keda-metrics-apiserver-77857b7b98-d4xmj\" (UID: \"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:07.686449 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.686420 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lkpb\" (UniqueName: \"kubernetes.io/projected/2f2baaa4-05b4-4efc-bc4e-476d4314d0e5-kube-api-access-8lkpb\") pod \"keda-admission-6c4d864b86-m5hrw\" (UID: \"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5\") " pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.837899 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.837864 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:07.916031 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:07.915999 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:08.119796 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.119718 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-m5hrw"] Sep 18 17:02:08.122669 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:02:08.122641 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2baaa4_05b4_4efc_bc4e_476d4314d0e5.slice/crio-b5283937a373ac1b5fcaf17830e5d617c0520f6e9f494182cf782e5b3f914910 WatchSource:0}: Error finding container b5283937a373ac1b5fcaf17830e5d617c0520f6e9f494182cf782e5b3f914910: Status 404 returned error can't find the container with id b5283937a373ac1b5fcaf17830e5d617c0520f6e9f494182cf782e5b3f914910 Sep 18 17:02:08.197624 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.197589 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj"] Sep 18 17:02:08.202437 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:02:08.202405 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a36ced1_64e8_47d6_b6ec_777cb1dc7ab4.slice/crio-598b0d3ad56e55ec77abcefec5c674a0181fd6b7cf2e88ce38e3d543e3b28cf2 WatchSource:0}: Error finding container 598b0d3ad56e55ec77abcefec5c674a0181fd6b7cf2e88ce38e3d543e3b28cf2: Status 404 returned error can't find the container with id 598b0d3ad56e55ec77abcefec5c674a0181fd6b7cf2e88ce38e3d543e3b28cf2 Sep 18 17:02:08.275964 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.275918 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:08.278549 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.278528 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7e70dd34-5b2c-4b87-bdb9-672855bf4bf4-certificates\") pod \"keda-operator-579c8958f7-mw86p\" (UID: \"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4\") " pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:08.317144 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.317107 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:08.466388 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.466346 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" event={"ID":"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5","Type":"ContainerStarted","Data":"b5283937a373ac1b5fcaf17830e5d617c0520f6e9f494182cf782e5b3f914910"} Sep 18 17:02:08.467771 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.467741 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" event={"ID":"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4","Type":"ContainerStarted","Data":"598b0d3ad56e55ec77abcefec5c674a0181fd6b7cf2e88ce38e3d543e3b28cf2"} Sep 18 17:02:08.599686 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:08.599661 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-mw86p"] Sep 18 17:02:08.602077 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:02:08.602051 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e70dd34_5b2c_4b87_bdb9_672855bf4bf4.slice/crio-c170760c6eed9d1941f691a35958dfb513c2b9ae6ff2ea21bb6399c995aab08d WatchSource:0}: Error finding container c170760c6eed9d1941f691a35958dfb513c2b9ae6ff2ea21bb6399c995aab08d: Status 404 returned error can't find the container with id c170760c6eed9d1941f691a35958dfb513c2b9ae6ff2ea21bb6399c995aab08d Sep 18 17:02:09.476163 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:09.476123 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-mw86p" event={"ID":"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4","Type":"ContainerStarted","Data":"c170760c6eed9d1941f691a35958dfb513c2b9ae6ff2ea21bb6399c995aab08d"} Sep 18 17:02:10.483043 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:10.482998 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" event={"ID":"2f2baaa4-05b4-4efc-bc4e-476d4314d0e5","Type":"ContainerStarted","Data":"5ef6278517cd39cd005778c046174dd7d2c3e0048214ec810a11eb2c4af22f46"} Sep 18 17:02:10.483514 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:10.483193 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:10.519710 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:10.519605 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" podStartSLOduration=2.217492927 podStartE2EDuration="3.519579261s" totalImagesPullingTime="1.302086334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:02:10.518732956 +0000 UTC m=+773.794858129" watchObservedRunningTime="2026-09-18 17:02:10.519579261 +0000 UTC m=+773.795704435" Sep 18 17:02:12.492945 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:12.492830 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-mw86p" event={"ID":"7e70dd34-5b2c-4b87-bdb9-672855bf4bf4","Type":"ContainerStarted","Data":"9e8abb7f78914b95ae9d677a10eb008775d89a87d3cc8751540255dd43e1ee21"} Sep 18 17:02:12.493350 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:12.492988 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:02:12.494398 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:12.494372 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" event={"ID":"6a36ced1-64e8-47d6-b6ec-777cb1dc7ab4","Type":"ContainerStarted","Data":"13ec10967043c5ce0c51455d51adc023edd995f386d38c964b7f03538ca8f92d"} Sep 18 17:02:12.494518 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:12.494425 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:12.516017 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:12.515956 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-mw86p" podStartSLOduration=2.981309186 podStartE2EDuration="6.515932378s" totalImagesPullingTime="3.534623192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:02:12.514471323 +0000 UTC m=+775.790596508" watchObservedRunningTime="2026-09-18 17:02:12.515932378 +0000 UTC m=+775.792057550" Sep 18 17:02:12.543532 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:12.543471 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" podStartSLOduration=2.612638295 podStartE2EDuration="6.543446036s" totalImagesPullingTime="3.930807741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:02:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:02:12.543095975 +0000 UTC m=+775.819221168" watchObservedRunningTime="2026-09-18 17:02:12.543446036 +0000 UTC m=+775.819571210" Sep 18 17:02:23.503482 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:23.503446 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-d4xmj" Sep 18 17:02:27.464798 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:27.464761 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-cp5fz" Sep 18 17:02:31.489818 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:31.489787 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-m5hrw" Sep 18 17:02:33.500259 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:02:33.500229 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-mw86p" Sep 18 17:03:28.110242 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.110202 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp"] Sep 18 17:03:28.112645 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.112622 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.115472 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.115448 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 18 17:03:28.117054 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.117030 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Sep 18 17:03:28.117184 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.117030 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 18 17:03:28.117184 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.117030 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-p7wt2\"" Sep 18 17:03:28.123772 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.123745 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp"] Sep 18 17:03:28.206280 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.206237 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8ddcf41-3ab3-4802-9c7d-3647a7abada9-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-cd9dp\" (UID: \"c8ddcf41-3ab3-4802-9c7d-3647a7abada9\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.206280 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.206278 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kxd\" (UniqueName: \"kubernetes.io/projected/c8ddcf41-3ab3-4802-9c7d-3647a7abada9-kube-api-access-96kxd\") pod \"llmisvc-controller-manager-645f9c8fd6-cd9dp\" (UID: \"c8ddcf41-3ab3-4802-9c7d-3647a7abada9\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.306912 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.306877 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8ddcf41-3ab3-4802-9c7d-3647a7abada9-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-cd9dp\" (UID: \"c8ddcf41-3ab3-4802-9c7d-3647a7abada9\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.306912 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.306914 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kxd\" (UniqueName: \"kubernetes.io/projected/c8ddcf41-3ab3-4802-9c7d-3647a7abada9-kube-api-access-96kxd\") pod \"llmisvc-controller-manager-645f9c8fd6-cd9dp\" (UID: \"c8ddcf41-3ab3-4802-9c7d-3647a7abada9\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.309491 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.309468 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8ddcf41-3ab3-4802-9c7d-3647a7abada9-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-cd9dp\" (UID: \"c8ddcf41-3ab3-4802-9c7d-3647a7abada9\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.319185 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.319149 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kxd\" (UniqueName: \"kubernetes.io/projected/c8ddcf41-3ab3-4802-9c7d-3647a7abada9-kube-api-access-96kxd\") pod \"llmisvc-controller-manager-645f9c8fd6-cd9dp\" (UID: \"c8ddcf41-3ab3-4802-9c7d-3647a7abada9\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.422585 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.422475 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:28.698296 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.698200 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp"] Sep 18 17:03:28.701525 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:03:28.701497 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc8ddcf41_3ab3_4802_9c7d_3647a7abada9.slice/crio-a1a787deed10bdbda8ecc2322edbfb430e43f9eccc5d1dfe1a69e9796401817a WatchSource:0}: Error finding container a1a787deed10bdbda8ecc2322edbfb430e43f9eccc5d1dfe1a69e9796401817a: Status 404 returned error can't find the container with id a1a787deed10bdbda8ecc2322edbfb430e43f9eccc5d1dfe1a69e9796401817a Sep 18 17:03:28.782360 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:28.782322 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" event={"ID":"c8ddcf41-3ab3-4802-9c7d-3647a7abada9","Type":"ContainerStarted","Data":"a1a787deed10bdbda8ecc2322edbfb430e43f9eccc5d1dfe1a69e9796401817a"} Sep 18 17:03:31.798758 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:31.798719 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" event={"ID":"c8ddcf41-3ab3-4802-9c7d-3647a7abada9","Type":"ContainerStarted","Data":"4a65970fb69b6263242ff4e29b4deee170cbb38c00390e65aea75b6fb58420a1"} Sep 18 17:03:31.799157 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:31.798862 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:03:31.816492 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:03:31.816438 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" podStartSLOduration=1.238047363 podStartE2EDuration="3.816421741s" totalImagesPullingTime="2.578374378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:03:31.815518257 +0000 UTC m=+855.091643443" watchObservedRunningTime="2026-09-18 17:03:31.816421741 +0000 UTC m=+855.092546965" Sep 18 17:04:02.805943 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:04:02.805873 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-cd9dp" Sep 18 17:04:18.839920 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:04:18.839891 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:04:18.841698 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:04:18.841681 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:04:18.862281 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:04:18.862256 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:04:18.864154 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:04:18.864123 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:06:33.192286 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.192248 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28"] Sep 18 17:06:33.195540 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.195517 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.198176 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.198150 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a4cbd-predictor-serving-cert\"" Sep 18 17:06:33.198176 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.198165 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a4cbd-kube-rbac-proxy-sar-config\"" Sep 18 17:06:33.198176 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.198184 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 18 17:06:33.199263 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.199247 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-97cfm\"" Sep 18 17:06:33.199356 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.199251 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 18 17:06:33.205691 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.205666 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28"] Sep 18 17:06:33.323345 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.323307 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/32e51e91-1f04-4a84-b209-0682fd629c94-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.323590 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.323372 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.323590 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.323401 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfblm\" (UniqueName: \"kubernetes.io/projected/32e51e91-1f04-4a84-b209-0682fd629c94-kube-api-access-hfblm\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.424621 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.424548 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.424805 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.424693 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfblm\" (UniqueName: \"kubernetes.io/projected/32e51e91-1f04-4a84-b209-0682fd629c94-kube-api-access-hfblm\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.424805 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:06:33.424717 2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-serving-cert: secret "error-404-isvc-a4cbd-predictor-serving-cert" not found Sep 18 17:06:33.424805 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.424746 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/32e51e91-1f04-4a84-b209-0682fd629c94-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.424931 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:06:33.424807 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls podName:32e51e91-1f04-4a84-b209-0682fd629c94 nodeName:}" failed. No retries permitted until 2026-09-18 17:06:33.924784751 +0000 UTC m=+1037.200909920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls") pod "error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" (UID: "32e51e91-1f04-4a84-b209-0682fd629c94") : secret "error-404-isvc-a4cbd-predictor-serving-cert" not found Sep 18 17:06:33.425357 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.425339 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/32e51e91-1f04-4a84-b209-0682fd629c94-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.436680 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.436647 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfblm\" (UniqueName: \"kubernetes.io/projected/32e51e91-1f04-4a84-b209-0682fd629c94-kube-api-access-hfblm\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.929138 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:33.929091 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:33.929322 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:06:33.929260 2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-serving-cert: secret "error-404-isvc-a4cbd-predictor-serving-cert" not found Sep 18 17:06:33.929367 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:06:33.929337 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls podName:32e51e91-1f04-4a84-b209-0682fd629c94 nodeName:}" failed. No retries permitted until 2026-09-18 17:06:34.92932144 +0000 UTC m=+1038.205446590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls") pod "error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" (UID: "32e51e91-1f04-4a84-b209-0682fd629c94") : secret "error-404-isvc-a4cbd-predictor-serving-cert" not found Sep 18 17:06:34.937495 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:34.937438 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:34.940175 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:34.940155 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") pod \"error-404-isvc-a4cbd-predictor-7d7d594689-8gj28\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:35.006455 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:35.006413 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:35.293055 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:35.292853 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28"] Sep 18 17:06:35.295709 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:06:35.295674 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e51e91_1f04_4a84_b209_0682fd629c94.slice/crio-756af10e82e6757fc1e0ce1be6837abec015c52133fc7e0ba1a8585f640557e6 WatchSource:0}: Error finding container 756af10e82e6757fc1e0ce1be6837abec015c52133fc7e0ba1a8585f640557e6: Status 404 returned error can't find the container with id 756af10e82e6757fc1e0ce1be6837abec015c52133fc7e0ba1a8585f640557e6 Sep 18 17:06:35.297455 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:35.297440 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 18 17:06:35.495145 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:35.495109 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" event={"ID":"32e51e91-1f04-4a84-b209-0682fd629c94","Type":"ContainerStarted","Data":"756af10e82e6757fc1e0ce1be6837abec015c52133fc7e0ba1a8585f640557e6"} Sep 18 17:06:48.562387 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:48.562345 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" event={"ID":"32e51e91-1f04-4a84-b209-0682fd629c94","Type":"ContainerStarted","Data":"05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49"} Sep 18 17:06:50.581245 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:50.581202 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" event={"ID":"32e51e91-1f04-4a84-b209-0682fd629c94","Type":"ContainerStarted","Data":"4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d"} Sep 18 17:06:50.581775 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:50.581430 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:50.581775 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:50.581490 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:50.582876 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:50.582848 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:06:50.604601 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:50.604521 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podStartSLOduration=2.942980727 podStartE2EDuration="17.604499871s" totalImagesPullingTime="14.661519144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:06:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:06:50.602441746 +0000 UTC m=+1053.878566939" watchObservedRunningTime="2026-09-18 17:06:50.604499871 +0000 UTC m=+1053.880625044" Sep 18 17:06:51.585568 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:51.585527 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:06:56.591097 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:56.591064 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:06:56.591759 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:06:56.591732 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:07:06.591897 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:07:06.591809 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:07:16.592743 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:07:16.592692 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:07:26.592463 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:07:26.592420 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:07:36.592597 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:07:36.592530 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:07:46.592402 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:07:46.592368 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:08:24.040890 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.040848 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28"] Sep 18 17:08:24.041527 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.041099 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" containerID="cri-o://05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49" gracePeriod=30 Sep 18 17:08:24.041527 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.041158 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kube-rbac-proxy" containerID="cri-o://4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d" gracePeriod=30 Sep 18 17:08:24.221040 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.221007 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788"] Sep 18 17:08:24.224008 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.223980 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.227670 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.227641 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1ff44-predictor-serving-cert\"" Sep 18 17:08:24.227820 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.227642 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-1ff44-kube-rbac-proxy-sar-config\"" Sep 18 17:08:24.260676 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.260634 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788"] Sep 18 17:08:24.394428 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.394395 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw4sp\" (UniqueName: \"kubernetes.io/projected/c3c8359d-7132-4959-b071-87f647ac270d-kube-api-access-pw4sp\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.394635 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.394449 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3c8359d-7132-4959-b071-87f647ac270d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.394635 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.394467 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3c8359d-7132-4959-b071-87f647ac270d-proxy-tls\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.495831 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.495790 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw4sp\" (UniqueName: \"kubernetes.io/projected/c3c8359d-7132-4959-b071-87f647ac270d-kube-api-access-pw4sp\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.496023 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.495845 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3c8359d-7132-4959-b071-87f647ac270d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.496023 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.495864 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3c8359d-7132-4959-b071-87f647ac270d-proxy-tls\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.496663 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.496631 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3c8359d-7132-4959-b071-87f647ac270d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.498634 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.498610 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3c8359d-7132-4959-b071-87f647ac270d-proxy-tls\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.510551 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.510519 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw4sp\" (UniqueName: \"kubernetes.io/projected/c3c8359d-7132-4959-b071-87f647ac270d-kube-api-access-pw4sp\") pod \"error-404-isvc-1ff44-predictor-66c97d8c59-4f788\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.534443 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.534399 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:24.823830 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.823617 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788"] Sep 18 17:08:24.826528 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:08:24.826497 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c8359d_7132_4959_b071_87f647ac270d.slice/crio-1ddd2e7af0594649b0aa85ea2368c795ff04d38ddcea77d28a9254366de64373 WatchSource:0}: Error finding container 1ddd2e7af0594649b0aa85ea2368c795ff04d38ddcea77d28a9254366de64373: Status 404 returned error can't find the container with id 1ddd2e7af0594649b0aa85ea2368c795ff04d38ddcea77d28a9254366de64373 Sep 18 17:08:24.943538 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.943501 2586 generic.go:356] "Generic (PLEG): container finished" podID="32e51e91-1f04-4a84-b209-0682fd629c94" containerID="4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d" exitCode=2 Sep 18 17:08:24.943754 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.943592 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" event={"ID":"32e51e91-1f04-4a84-b209-0682fd629c94","Type":"ContainerDied","Data":"4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d"} Sep 18 17:08:24.945498 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.945458 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" event={"ID":"c3c8359d-7132-4959-b071-87f647ac270d","Type":"ContainerStarted","Data":"b3b3c6a7f723b757116738813af30e623afd5d528812a8bc04fca5d33f94abcb"} Sep 18 17:08:24.945693 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:24.945541 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" event={"ID":"c3c8359d-7132-4959-b071-87f647ac270d","Type":"ContainerStarted","Data":"1ddd2e7af0594649b0aa85ea2368c795ff04d38ddcea77d28a9254366de64373"} Sep 18 17:08:25.952020 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:25.951969 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" event={"ID":"c3c8359d-7132-4959-b071-87f647ac270d","Type":"ContainerStarted","Data":"505dfed2b296036a3853306c208bff2102e73334b1da0f971490d781ffac1fe0"} Sep 18 17:08:25.952529 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:25.952068 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:25.979710 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:25.979653 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podStartSLOduration=1.9796342820000001 podStartE2EDuration="1.979634282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:08:25.979510317 +0000 UTC m=+1149.255635506" watchObservedRunningTime="2026-09-18 17:08:25.979634282 +0000 UTC m=+1149.255759455" Sep 18 17:08:26.586481 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:26.586432 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.20:8643/healthz\": dial tcp 10.134.0.20:8643: connect: connection refused" Sep 18 17:08:26.591875 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:26.591838 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused" Sep 18 17:08:26.956596 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:26.956490 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:26.958019 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:26.957989 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:08:27.887831 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.887803 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:08:27.917056 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.916969 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/32e51e91-1f04-4a84-b209-0682fd629c94-kube-rbac-proxy-sar-config\") pod \"32e51e91-1f04-4a84-b209-0682fd629c94\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " Sep 18 17:08:27.917056 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.917027 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfblm\" (UniqueName: \"kubernetes.io/projected/32e51e91-1f04-4a84-b209-0682fd629c94-kube-api-access-hfblm\") pod \"32e51e91-1f04-4a84-b209-0682fd629c94\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " Sep 18 17:08:27.917056 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.917048 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") pod \"32e51e91-1f04-4a84-b209-0682fd629c94\" (UID: \"32e51e91-1f04-4a84-b209-0682fd629c94\") " Sep 18 17:08:27.917419 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.917388 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32e51e91-1f04-4a84-b209-0682fd629c94-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "32e51e91-1f04-4a84-b209-0682fd629c94" (UID: "32e51e91-1f04-4a84-b209-0682fd629c94"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:08:27.919425 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.919395 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "32e51e91-1f04-4a84-b209-0682fd629c94" (UID: "32e51e91-1f04-4a84-b209-0682fd629c94"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:08:27.919634 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.919598 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e51e91-1f04-4a84-b209-0682fd629c94-kube-api-access-hfblm" (OuterVolumeSpecName: "kube-api-access-hfblm") pod "32e51e91-1f04-4a84-b209-0682fd629c94" (UID: "32e51e91-1f04-4a84-b209-0682fd629c94"). InnerVolumeSpecName "kube-api-access-hfblm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:08:27.961342 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.961308 2586 generic.go:356] "Generic (PLEG): container finished" podID="32e51e91-1f04-4a84-b209-0682fd629c94" containerID="05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49" exitCode=0 Sep 18 17:08:27.961801 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.961365 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" Sep 18 17:08:27.961801 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.961382 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" event={"ID":"32e51e91-1f04-4a84-b209-0682fd629c94","Type":"ContainerDied","Data":"05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49"} Sep 18 17:08:27.961801 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.961453 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28" event={"ID":"32e51e91-1f04-4a84-b209-0682fd629c94","Type":"ContainerDied","Data":"756af10e82e6757fc1e0ce1be6837abec015c52133fc7e0ba1a8585f640557e6"} Sep 18 17:08:27.961801 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.961493 2586 scope.go:117] "RemoveContainer" containerID="4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d" Sep 18 17:08:27.962012 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.961839 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:08:27.970986 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.970967 2586 scope.go:117] "RemoveContainer" containerID="05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49" Sep 18 17:08:27.979084 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.979064 2586 scope.go:117] "RemoveContainer" containerID="4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d" Sep 18 17:08:27.979385 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:08:27.979365 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d\": container with ID starting with 4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d not found: ID does not exist" containerID="4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d" Sep 18 17:08:27.979433 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.979397 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d"} err="failed to get container status \"4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d\": rpc error: code = NotFound desc = could not find container \"4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d\": container with ID starting with 4e759e6a4602fd1c71fcc24f727a932b49186f2607850be2e7acab36fc89249d not found: ID does not exist" Sep 18 17:08:27.979433 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.979417 2586 scope.go:117] "RemoveContainer" containerID="05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49" Sep 18 17:08:27.979697 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:08:27.979679 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49\": container with ID starting with 05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49 not found: ID does not exist" containerID="05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49" Sep 18 17:08:27.979742 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:27.979704 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49"} err="failed to get container status \"05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49\": rpc error: code = NotFound desc = could not find container \"05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49\": container with ID starting with 05e51a314c952c4c93b95eda0ca34e73dc45df6ffe6fcffbb9f4197288358b49 not found: ID does not exist" Sep 18 17:08:28.014951 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:28.014907 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28"] Sep 18 17:08:28.017989 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:28.017961 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/32e51e91-1f04-4a84-b209-0682fd629c94-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:08:28.018108 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:28.017994 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfblm\" (UniqueName: \"kubernetes.io/projected/32e51e91-1f04-4a84-b209-0682fd629c94-kube-api-access-hfblm\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:08:28.018108 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:28.018085 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/32e51e91-1f04-4a84-b209-0682fd629c94-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:08:28.022801 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:28.022773 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a4cbd-predictor-7d7d594689-8gj28"] Sep 18 17:08:28.915055 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:28.914978 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" path="/var/lib/kubelet/pods/32e51e91-1f04-4a84-b209-0682fd629c94/volumes" Sep 18 17:08:32.966883 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:32.966845 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:08:32.967502 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:32.967473 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:08:42.967679 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:42.967632 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:08:52.968432 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:52.968394 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:08:54.675571 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.675513 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw"] Sep 18 17:08:54.676076 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.675984 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" Sep 18 17:08:54.676076 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.676003 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" Sep 18 17:08:54.676076 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.676022 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kube-rbac-proxy" Sep 18 17:08:54.676076 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.676029 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kube-rbac-proxy" Sep 18 17:08:54.676288 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.676089 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kserve-container" Sep 18 17:08:54.676288 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.676100 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e51e91-1f04-4a84-b209-0682fd629c94" containerName="kube-rbac-proxy" Sep 18 17:08:54.679098 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.679076 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.683345 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.683322 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9846a-predictor-serving-cert\"" Sep 18 17:08:54.683712 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.683693 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-9846a-kube-rbac-proxy-sar-config\"" Sep 18 17:08:54.696477 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.696445 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw"] Sep 18 17:08:54.698595 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.698569 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqw55\" (UniqueName: \"kubernetes.io/projected/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-api-access-jqw55\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.698740 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.698622 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.698740 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.698671 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.799176 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.799128 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.799373 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.799195 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.799373 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.799219 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqw55\" (UniqueName: \"kubernetes.io/projected/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-api-access-jqw55\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.799373 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:08:54.799340 2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-9846a-predictor-serving-cert: secret "error-404-isvc-9846a-predictor-serving-cert" not found Sep 18 17:08:54.799492 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:08:54.799422 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls podName:f8fc418d-fb75-41ce-a550-2e6ff248ac95 nodeName:}" failed. No retries permitted until 2026-09-18 17:08:55.299402493 +0000 UTC m=+1178.575527659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls") pod "error-404-isvc-9846a-predictor-797cb6675c-5kzcw" (UID: "f8fc418d-fb75-41ce-a550-2e6ff248ac95") : secret "error-404-isvc-9846a-predictor-serving-cert" not found Sep 18 17:08:54.799884 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.799866 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:54.809926 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:54.809899 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqw55\" (UniqueName: \"kubernetes.io/projected/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-api-access-jqw55\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:55.302903 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:55.302870 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:55.305722 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:55.305695 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls\") pod \"error-404-isvc-9846a-predictor-797cb6675c-5kzcw\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:55.588732 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:55.588697 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:55.866159 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:55.866136 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw"] Sep 18 17:08:55.867859 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:08:55.867831 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fc418d_fb75_41ce_a550_2e6ff248ac95.slice/crio-592adddc7b9a585b63d5117eb478684f4c9288250624e35859c6fa6a332c56b1 WatchSource:0}: Error finding container 592adddc7b9a585b63d5117eb478684f4c9288250624e35859c6fa6a332c56b1: Status 404 returned error can't find the container with id 592adddc7b9a585b63d5117eb478684f4c9288250624e35859c6fa6a332c56b1 Sep 18 17:08:56.080252 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:56.080208 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" event={"ID":"f8fc418d-fb75-41ce-a550-2e6ff248ac95","Type":"ContainerStarted","Data":"d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80"} Sep 18 17:08:56.080464 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:56.080272 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" event={"ID":"f8fc418d-fb75-41ce-a550-2e6ff248ac95","Type":"ContainerStarted","Data":"a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b"} Sep 18 17:08:56.080464 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:56.080293 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" event={"ID":"f8fc418d-fb75-41ce-a550-2e6ff248ac95","Type":"ContainerStarted","Data":"592adddc7b9a585b63d5117eb478684f4c9288250624e35859c6fa6a332c56b1"} Sep 18 17:08:56.080464 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:56.080417 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:56.104203 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:56.104149 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podStartSLOduration=2.104131817 podStartE2EDuration="2.104131817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:08:56.10348638 +0000 UTC m=+1179.379611584" watchObservedRunningTime="2026-09-18 17:08:56.104131817 +0000 UTC m=+1179.380256990" Sep 18 17:08:57.084784 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:57.084751 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:08:57.086208 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:57.086181 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:08:58.088462 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:08:58.088416 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:09:02.967829 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:02.967784 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:09:03.094271 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:03.094241 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:09:03.094882 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:03.094853 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:09:12.967747 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:12.967694 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:09:13.095164 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:13.095114 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:09:18.865047 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:18.865019 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:09:18.866877 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:18.866856 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:09:18.884802 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:18.884771 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:09:18.886920 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:18.886899 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:09:22.968742 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:22.968708 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:09:23.095113 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:23.095074 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:09:33.095122 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:33.095077 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:09:43.095787 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:43.095734 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:09:53.096300 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:09:53.096270 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:14:18.889599 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:14:18.889572 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:14:18.891588 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:14:18.891548 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:14:18.907012 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:14:18.906986 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:14:18.908802 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:14:18.908781 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:17:49.436896 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.436819 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788"] Sep 18 17:17:49.439482 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.437071 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" containerID="cri-o://b3b3c6a7f723b757116738813af30e623afd5d528812a8bc04fca5d33f94abcb" gracePeriod=30 Sep 18 17:17:49.439482 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.437118 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kube-rbac-proxy" containerID="cri-o://505dfed2b296036a3853306c208bff2102e73334b1da0f971490d781ffac1fe0" gracePeriod=30 Sep 18 17:17:49.553892 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.553857 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8"] Sep 18 17:17:49.556888 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.556870 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.559383 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.559356 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-21436-predictor-serving-cert\"" Sep 18 17:17:49.559893 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.559877 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-21436-kube-rbac-proxy-sar-config\"" Sep 18 17:17:49.566885 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.566849 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8"] Sep 18 17:17:49.670232 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.670195 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30a227ef-983f-44fe-95ec-94656ce728f5-proxy-tls\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.670414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.670266 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4vhk\" (UniqueName: \"kubernetes.io/projected/30a227ef-983f-44fe-95ec-94656ce728f5-kube-api-access-x4vhk\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.670414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.670298 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30a227ef-983f-44fe-95ec-94656ce728f5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.771628 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.771522 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4vhk\" (UniqueName: \"kubernetes.io/projected/30a227ef-983f-44fe-95ec-94656ce728f5-kube-api-access-x4vhk\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.771628 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.771591 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30a227ef-983f-44fe-95ec-94656ce728f5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.771954 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.771654 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30a227ef-983f-44fe-95ec-94656ce728f5-proxy-tls\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.772309 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.772282 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30a227ef-983f-44fe-95ec-94656ce728f5-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.774417 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.774393 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30a227ef-983f-44fe-95ec-94656ce728f5-proxy-tls\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.782301 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.782271 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4vhk\" (UniqueName: \"kubernetes.io/projected/30a227ef-983f-44fe-95ec-94656ce728f5-kube-api-access-x4vhk\") pod \"error-404-isvc-21436-predictor-8574dbfbf7-mdtp8\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:49.867185 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:49.867140 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:50.140489 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:50.140461 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8"] Sep 18 17:17:50.142952 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:17:50.142921 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a227ef_983f_44fe_95ec_94656ce728f5.slice/crio-cf2070e4d2969cc6806a612f1ba7a35d7ce151c471cabaabd91d899d32636bcb WatchSource:0}: Error finding container cf2070e4d2969cc6806a612f1ba7a35d7ce151c471cabaabd91d899d32636bcb: Status 404 returned error can't find the container with id cf2070e4d2969cc6806a612f1ba7a35d7ce151c471cabaabd91d899d32636bcb Sep 18 17:17:50.166452 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:50.166420 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" event={"ID":"30a227ef-983f-44fe-95ec-94656ce728f5","Type":"ContainerStarted","Data":"cf2070e4d2969cc6806a612f1ba7a35d7ce151c471cabaabd91d899d32636bcb"} Sep 18 17:17:50.168275 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:50.168247 2586 generic.go:356] "Generic (PLEG): container finished" podID="c3c8359d-7132-4959-b071-87f647ac270d" containerID="505dfed2b296036a3853306c208bff2102e73334b1da0f971490d781ffac1fe0" exitCode=2 Sep 18 17:17:50.168404 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:50.168307 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" event={"ID":"c3c8359d-7132-4959-b071-87f647ac270d","Type":"ContainerDied","Data":"505dfed2b296036a3853306c208bff2102e73334b1da0f971490d781ffac1fe0"} Sep 18 17:17:51.174547 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:51.174515 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" event={"ID":"30a227ef-983f-44fe-95ec-94656ce728f5","Type":"ContainerStarted","Data":"78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914"} Sep 18 17:17:51.174957 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:51.174597 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" event={"ID":"30a227ef-983f-44fe-95ec-94656ce728f5","Type":"ContainerStarted","Data":"28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21"} Sep 18 17:17:51.174957 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:51.174658 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:51.193950 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:51.193907 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podStartSLOduration=2.193892641 podStartE2EDuration="2.193892641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:17:51.192247807 +0000 UTC m=+1714.468372995" watchObservedRunningTime="2026-09-18 17:17:51.193892641 +0000 UTC m=+1714.470017871" Sep 18 17:17:52.179047 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:52.179007 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:52.180245 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:52.180224 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:17:52.962373 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:52.962339 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused" Sep 18 17:17:52.967752 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:52.967727 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused" Sep 18 17:17:53.184870 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.184838 2586 generic.go:356] "Generic (PLEG): container finished" podID="c3c8359d-7132-4959-b071-87f647ac270d" containerID="b3b3c6a7f723b757116738813af30e623afd5d528812a8bc04fca5d33f94abcb" exitCode=0 Sep 18 17:17:53.185222 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.184911 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" event={"ID":"c3c8359d-7132-4959-b071-87f647ac270d","Type":"ContainerDied","Data":"b3b3c6a7f723b757116738813af30e623afd5d528812a8bc04fca5d33f94abcb"} Sep 18 17:17:53.185222 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.184983 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" event={"ID":"c3c8359d-7132-4959-b071-87f647ac270d","Type":"ContainerDied","Data":"1ddd2e7af0594649b0aa85ea2368c795ff04d38ddcea77d28a9254366de64373"} Sep 18 17:17:53.185222 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.185014 2586 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ddd2e7af0594649b0aa85ea2368c795ff04d38ddcea77d28a9254366de64373" Sep 18 17:17:53.185396 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.185231 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:17:53.192526 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.192502 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:17:53.300367 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.300273 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3c8359d-7132-4959-b071-87f647ac270d-proxy-tls\") pod \"c3c8359d-7132-4959-b071-87f647ac270d\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " Sep 18 17:17:53.300367 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.300329 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3c8359d-7132-4959-b071-87f647ac270d-kube-rbac-proxy-sar-config\") pod \"c3c8359d-7132-4959-b071-87f647ac270d\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " Sep 18 17:17:53.300367 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.300349 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw4sp\" (UniqueName: \"kubernetes.io/projected/c3c8359d-7132-4959-b071-87f647ac270d-kube-api-access-pw4sp\") pod \"c3c8359d-7132-4959-b071-87f647ac270d\" (UID: \"c3c8359d-7132-4959-b071-87f647ac270d\") " Sep 18 17:17:53.300780 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.300752 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3c8359d-7132-4959-b071-87f647ac270d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c3c8359d-7132-4959-b071-87f647ac270d" (UID: "c3c8359d-7132-4959-b071-87f647ac270d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:17:53.302682 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.302662 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3c8359d-7132-4959-b071-87f647ac270d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c3c8359d-7132-4959-b071-87f647ac270d" (UID: "c3c8359d-7132-4959-b071-87f647ac270d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:17:53.302682 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.302670 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c8359d-7132-4959-b071-87f647ac270d-kube-api-access-pw4sp" (OuterVolumeSpecName: "kube-api-access-pw4sp") pod "c3c8359d-7132-4959-b071-87f647ac270d" (UID: "c3c8359d-7132-4959-b071-87f647ac270d"). InnerVolumeSpecName "kube-api-access-pw4sp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:17:53.401236 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.401191 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c3c8359d-7132-4959-b071-87f647ac270d-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:17:53.401236 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.401227 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c3c8359d-7132-4959-b071-87f647ac270d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:17:53.401236 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:53.401238 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw4sp\" (UniqueName: \"kubernetes.io/projected/c3c8359d-7132-4959-b071-87f647ac270d-kube-api-access-pw4sp\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:17:54.189153 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:54.189121 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788" Sep 18 17:17:54.216017 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:54.213256 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788"] Sep 18 17:17:54.216893 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:54.216867 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-1ff44-predictor-66c97d8c59-4f788"] Sep 18 17:17:54.915798 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:54.915761 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3c8359d-7132-4959-b071-87f647ac270d" path="/var/lib/kubelet/pods/c3c8359d-7132-4959-b071-87f647ac270d/volumes" Sep 18 17:17:58.189729 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:58.189699 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:17:58.190149 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:17:58.190125 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:18:08.190200 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:08.190151 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:18:18.190283 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:18.190242 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:18:18.938853 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:18.938822 2586 scope.go:117] "RemoveContainer" containerID="b3b3c6a7f723b757116738813af30e623afd5d528812a8bc04fca5d33f94abcb" Sep 18 17:18:18.946898 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:18.946874 2586 scope.go:117] "RemoveContainer" containerID="505dfed2b296036a3853306c208bff2102e73334b1da0f971490d781ffac1fe0" Sep 18 17:18:20.057353 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.057316 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw"] Sep 18 17:18:20.057809 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.057577 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" containerID="cri-o://a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b" gracePeriod=30 Sep 18 17:18:20.057809 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.057629 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kube-rbac-proxy" containerID="cri-o://d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80" gracePeriod=30 Sep 18 17:18:20.268226 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268190 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v"] Sep 18 17:18:20.268484 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268473 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" Sep 18 17:18:20.268529 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268486 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" Sep 18 17:18:20.268529 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268497 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kube-rbac-proxy" Sep 18 17:18:20.268529 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268503 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kube-rbac-proxy" Sep 18 17:18:20.268640 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268550 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kube-rbac-proxy" Sep 18 17:18:20.268640 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.268577 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c8359d-7132-4959-b071-87f647ac270d" containerName="kserve-container" Sep 18 17:18:20.271441 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.271424 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.274718 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.274693 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-78e6b-kube-rbac-proxy-sar-config\"" Sep 18 17:18:20.274958 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.274922 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-78e6b-predictor-serving-cert\"" Sep 18 17:18:20.283348 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.283318 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v"] Sep 18 17:18:20.298395 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.298359 2586 generic.go:356] "Generic (PLEG): container finished" podID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerID="d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80" exitCode=2 Sep 18 17:18:20.298580 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.298435 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" event={"ID":"f8fc418d-fb75-41ce-a550-2e6ff248ac95","Type":"ContainerDied","Data":"d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80"} Sep 18 17:18:20.405340 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.405293 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlgvg\" (UniqueName: \"kubernetes.io/projected/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-api-access-qlgvg\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.405340 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.405342 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.405670 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.405414 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/792a1fde-2c89-425b-83f2-42d73a9c9317-proxy-tls\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.506257 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.506213 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgvg\" (UniqueName: \"kubernetes.io/projected/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-api-access-qlgvg\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.506257 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.506259 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.506501 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.506281 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/792a1fde-2c89-425b-83f2-42d73a9c9317-proxy-tls\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.506919 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.506898 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.508866 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.508844 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/792a1fde-2c89-425b-83f2-42d73a9c9317-proxy-tls\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.515901 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.515879 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlgvg\" (UniqueName: \"kubernetes.io/projected/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-api-access-qlgvg\") pod \"error-404-isvc-78e6b-predictor-547866bf4f-ss64v\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.587107 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.587068 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:20.863534 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:20.863508 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v"] Sep 18 17:18:20.865757 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:18:20.865732 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792a1fde_2c89_425b_83f2_42d73a9c9317.slice/crio-30104fc10b8bb8069c14f0a44bab9bc324b55ced48dcd68e1a81adbfa9eb6be3 WatchSource:0}: Error finding container 30104fc10b8bb8069c14f0a44bab9bc324b55ced48dcd68e1a81adbfa9eb6be3: Status 404 returned error can't find the container with id 30104fc10b8bb8069c14f0a44bab9bc324b55ced48dcd68e1a81adbfa9eb6be3 Sep 18 17:18:21.303901 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.303808 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" event={"ID":"792a1fde-2c89-425b-83f2-42d73a9c9317","Type":"ContainerStarted","Data":"bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6"} Sep 18 17:18:21.303901 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.303869 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" event={"ID":"792a1fde-2c89-425b-83f2-42d73a9c9317","Type":"ContainerStarted","Data":"ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a"} Sep 18 17:18:21.303901 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.303897 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:21.304329 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.303917 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" event={"ID":"792a1fde-2c89-425b-83f2-42d73a9c9317","Type":"ContainerStarted","Data":"30104fc10b8bb8069c14f0a44bab9bc324b55ced48dcd68e1a81adbfa9eb6be3"} Sep 18 17:18:21.304329 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.303957 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:21.305455 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.305428 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:18:21.324630 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:21.324580 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podStartSLOduration=1.324540875 podStartE2EDuration="1.324540875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:18:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:18:21.322970942 +0000 UTC m=+1744.599096135" watchObservedRunningTime="2026-09-18 17:18:21.324540875 +0000 UTC m=+1744.600666048" Sep 18 17:18:22.307956 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:22.307917 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:18:23.089447 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.089403 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.22:8643/healthz\": dial tcp 10.134.0.22:8643: connect: connection refused" Sep 18 17:18:23.094850 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.094811 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused" Sep 18 17:18:23.802820 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.802788 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:18:23.934035 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.933936 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqw55\" (UniqueName: \"kubernetes.io/projected/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-api-access-jqw55\") pod \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " Sep 18 17:18:23.934035 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.933984 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-rbac-proxy-sar-config\") pod \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " Sep 18 17:18:23.934035 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.934041 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls\") pod \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\" (UID: \"f8fc418d-fb75-41ce-a550-2e6ff248ac95\") " Sep 18 17:18:23.934366 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.934342 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f8fc418d-fb75-41ce-a550-2e6ff248ac95" (UID: "f8fc418d-fb75-41ce-a550-2e6ff248ac95"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:18:23.936372 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.936345 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f8fc418d-fb75-41ce-a550-2e6ff248ac95" (UID: "f8fc418d-fb75-41ce-a550-2e6ff248ac95"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:18:23.936458 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:23.936392 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-api-access-jqw55" (OuterVolumeSpecName: "kube-api-access-jqw55") pod "f8fc418d-fb75-41ce-a550-2e6ff248ac95" (UID: "f8fc418d-fb75-41ce-a550-2e6ff248ac95"). InnerVolumeSpecName "kube-api-access-jqw55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:18:24.035320 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.035275 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqw55\" (UniqueName: \"kubernetes.io/projected/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-api-access-jqw55\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:18:24.035320 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.035310 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f8fc418d-fb75-41ce-a550-2e6ff248ac95-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:18:24.035320 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.035322 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f8fc418d-fb75-41ce-a550-2e6ff248ac95-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:18:24.318296 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.318211 2586 generic.go:356] "Generic (PLEG): container finished" podID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerID="a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b" exitCode=0 Sep 18 17:18:24.318296 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.318257 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" event={"ID":"f8fc418d-fb75-41ce-a550-2e6ff248ac95","Type":"ContainerDied","Data":"a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b"} Sep 18 17:18:24.318296 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.318264 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" Sep 18 17:18:24.318571 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.318308 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw" event={"ID":"f8fc418d-fb75-41ce-a550-2e6ff248ac95","Type":"ContainerDied","Data":"592adddc7b9a585b63d5117eb478684f4c9288250624e35859c6fa6a332c56b1"} Sep 18 17:18:24.318571 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.318335 2586 scope.go:117] "RemoveContainer" containerID="d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80" Sep 18 17:18:24.327281 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.327255 2586 scope.go:117] "RemoveContainer" containerID="a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b" Sep 18 17:18:24.335533 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.335507 2586 scope.go:117] "RemoveContainer" containerID="d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80" Sep 18 17:18:24.335899 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:18:24.335878 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80\": container with ID starting with d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80 not found: ID does not exist" containerID="d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80" Sep 18 17:18:24.335988 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.335910 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80"} err="failed to get container status \"d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80\": rpc error: code = NotFound desc = could not find container \"d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80\": container with ID starting with d59b845991bc4cab121bd2b2eef52524a0a50e20d9bd3cd5544cde5b936d5c80 not found: ID does not exist" Sep 18 17:18:24.335988 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.335927 2586 scope.go:117] "RemoveContainer" containerID="a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b" Sep 18 17:18:24.336198 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:18:24.336178 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b\": container with ID starting with a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b not found: ID does not exist" containerID="a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b" Sep 18 17:18:24.336261 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.336208 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b"} err="failed to get container status \"a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b\": rpc error: code = NotFound desc = could not find container \"a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b\": container with ID starting with a42ed0016e8a7315d792b19dca014e32c5fe8659267a504a1a81ede3d8d8963b not found: ID does not exist" Sep 18 17:18:24.344788 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.344749 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw"] Sep 18 17:18:24.348236 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.348195 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-9846a-predictor-797cb6675c-5kzcw"] Sep 18 17:18:24.915621 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:24.915549 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" path="/var/lib/kubelet/pods/f8fc418d-fb75-41ce-a550-2e6ff248ac95/volumes" Sep 18 17:18:27.312782 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:27.312755 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:18:27.313320 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:27.313291 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:18:28.190889 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:28.190843 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:18:37.313729 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:37.313676 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:18:38.190867 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:38.190820 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused" Sep 18 17:18:47.313927 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:47.313882 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:18:48.191528 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:48.191498 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:18:57.313370 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:18:57.313329 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:19:07.314184 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:07.314139 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused" Sep 18 17:19:10.520006 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.519970 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8"] Sep 18 17:19:10.520460 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.520231 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" containerID="cri-o://28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21" gracePeriod=30 Sep 18 17:19:10.520460 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.520265 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kube-rbac-proxy" containerID="cri-o://78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914" gracePeriod=30 Sep 18 17:19:10.692200 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692167 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh"] Sep 18 17:19:10.692618 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692593 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" Sep 18 17:19:10.692776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692632 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" Sep 18 17:19:10.692776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692648 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kube-rbac-proxy" Sep 18 17:19:10.692776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692656 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kube-rbac-proxy" Sep 18 17:19:10.692776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692750 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kube-rbac-proxy" Sep 18 17:19:10.692776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.692763 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8fc418d-fb75-41ce-a550-2e6ff248ac95" containerName="kserve-container" Sep 18 17:19:10.695737 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.695716 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.699520 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.699499 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-cca0b-predictor-serving-cert\"" Sep 18 17:19:10.699780 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.699763 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-cca0b-kube-rbac-proxy-sar-config\"" Sep 18 17:19:10.719289 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.719254 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh"] Sep 18 17:19:10.796111 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.796001 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.796111 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.796050 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df76c216-e80b-41f0-bc1b-1384dc69cb83-proxy-tls\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.796334 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.796164 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5v8s\" (UniqueName: \"kubernetes.io/projected/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-api-access-t5v8s\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.897350 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.897310 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df76c216-e80b-41f0-bc1b-1384dc69cb83-proxy-tls\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.897534 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.897370 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5v8s\" (UniqueName: \"kubernetes.io/projected/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-api-access-t5v8s\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.897534 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.897399 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.898163 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.898136 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.899968 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.899939 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df76c216-e80b-41f0-bc1b-1384dc69cb83-proxy-tls\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:10.916229 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:10.916194 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5v8s\" (UniqueName: \"kubernetes.io/projected/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-api-access-t5v8s\") pod \"error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:11.006126 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.006081 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:11.292053 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.292018 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh"] Sep 18 17:19:11.294970 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:19:11.294939 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf76c216_e80b_41f0_bc1b_1384dc69cb83.slice/crio-e751e8a23f22c68f9d1fcdc316946acd3c143beede8db13ea87f68d2dcc51927 WatchSource:0}: Error finding container e751e8a23f22c68f9d1fcdc316946acd3c143beede8db13ea87f68d2dcc51927: Status 404 returned error can't find the container with id e751e8a23f22c68f9d1fcdc316946acd3c143beede8db13ea87f68d2dcc51927 Sep 18 17:19:11.524088 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.524037 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" event={"ID":"df76c216-e80b-41f0-bc1b-1384dc69cb83","Type":"ContainerStarted","Data":"9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95"} Sep 18 17:19:11.524521 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.524124 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" event={"ID":"df76c216-e80b-41f0-bc1b-1384dc69cb83","Type":"ContainerStarted","Data":"d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86"} Sep 18 17:19:11.524521 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.524153 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" event={"ID":"df76c216-e80b-41f0-bc1b-1384dc69cb83","Type":"ContainerStarted","Data":"e751e8a23f22c68f9d1fcdc316946acd3c143beede8db13ea87f68d2dcc51927"} Sep 18 17:19:11.524521 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.524295 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:11.525951 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.525922 2586 generic.go:356] "Generic (PLEG): container finished" podID="30a227ef-983f-44fe-95ec-94656ce728f5" containerID="78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914" exitCode=2 Sep 18 17:19:11.526044 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.525962 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" event={"ID":"30a227ef-983f-44fe-95ec-94656ce728f5","Type":"ContainerDied","Data":"78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914"} Sep 18 17:19:11.550965 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:11.550891 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podStartSLOduration=1.550866311 podStartE2EDuration="1.550866311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:19:11.547942522 +0000 UTC m=+1794.824067717" watchObservedRunningTime="2026-09-18 17:19:11.550866311 +0000 UTC m=+1794.826991486" Sep 18 17:19:12.530070 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:12.530038 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:12.531389 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:12.531362 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:13.185884 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:13.185842 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.23:8643/healthz\": dial tcp 10.134.0.23:8643: connect: connection refused" Sep 18 17:19:13.534723 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:13.534632 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:14.368457 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.368431 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:19:14.425628 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.425507 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4vhk\" (UniqueName: \"kubernetes.io/projected/30a227ef-983f-44fe-95ec-94656ce728f5-kube-api-access-x4vhk\") pod \"30a227ef-983f-44fe-95ec-94656ce728f5\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " Sep 18 17:19:14.425786 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.425640 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30a227ef-983f-44fe-95ec-94656ce728f5-proxy-tls\") pod \"30a227ef-983f-44fe-95ec-94656ce728f5\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " Sep 18 17:19:14.425786 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.425672 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30a227ef-983f-44fe-95ec-94656ce728f5-kube-rbac-proxy-sar-config\") pod \"30a227ef-983f-44fe-95ec-94656ce728f5\" (UID: \"30a227ef-983f-44fe-95ec-94656ce728f5\") " Sep 18 17:19:14.426070 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.426033 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a227ef-983f-44fe-95ec-94656ce728f5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "30a227ef-983f-44fe-95ec-94656ce728f5" (UID: "30a227ef-983f-44fe-95ec-94656ce728f5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:19:14.427905 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.427882 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a227ef-983f-44fe-95ec-94656ce728f5-kube-api-access-x4vhk" (OuterVolumeSpecName: "kube-api-access-x4vhk") pod "30a227ef-983f-44fe-95ec-94656ce728f5" (UID: "30a227ef-983f-44fe-95ec-94656ce728f5"). InnerVolumeSpecName "kube-api-access-x4vhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:19:14.428030 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.427908 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a227ef-983f-44fe-95ec-94656ce728f5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "30a227ef-983f-44fe-95ec-94656ce728f5" (UID: "30a227ef-983f-44fe-95ec-94656ce728f5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:19:14.526453 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.526411 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30a227ef-983f-44fe-95ec-94656ce728f5-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:19:14.526453 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.526446 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30a227ef-983f-44fe-95ec-94656ce728f5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:19:14.526453 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.526455 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4vhk\" (UniqueName: \"kubernetes.io/projected/30a227ef-983f-44fe-95ec-94656ce728f5-kube-api-access-x4vhk\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:19:14.539685 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.539651 2586 generic.go:356] "Generic (PLEG): container finished" podID="30a227ef-983f-44fe-95ec-94656ce728f5" containerID="28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21" exitCode=0 Sep 18 17:19:14.540105 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.539694 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" event={"ID":"30a227ef-983f-44fe-95ec-94656ce728f5","Type":"ContainerDied","Data":"28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21"} Sep 18 17:19:14.540105 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.539704 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" Sep 18 17:19:14.540105 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.539758 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8" event={"ID":"30a227ef-983f-44fe-95ec-94656ce728f5","Type":"ContainerDied","Data":"cf2070e4d2969cc6806a612f1ba7a35d7ce151c471cabaabd91d899d32636bcb"} Sep 18 17:19:14.540105 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.539799 2586 scope.go:117] "RemoveContainer" containerID="78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914" Sep 18 17:19:14.553316 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.553291 2586 scope.go:117] "RemoveContainer" containerID="28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21" Sep 18 17:19:14.561602 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.561550 2586 scope.go:117] "RemoveContainer" containerID="78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914" Sep 18 17:19:14.562004 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:19:14.561964 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914\": container with ID starting with 78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914 not found: ID does not exist" containerID="78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914" Sep 18 17:19:14.562062 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.562014 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914"} err="failed to get container status \"78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914\": rpc error: code = NotFound desc = could not find container \"78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914\": container with ID starting with 78aa5a3c12d7f4f61e00fd6910f0066df908ebdb13c72c53591744818b972914 not found: ID does not exist" Sep 18 17:19:14.562062 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.562034 2586 scope.go:117] "RemoveContainer" containerID="28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21" Sep 18 17:19:14.562301 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:19:14.562278 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21\": container with ID starting with 28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21 not found: ID does not exist" containerID="28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21" Sep 18 17:19:14.562340 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.562310 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21"} err="failed to get container status \"28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21\": rpc error: code = NotFound desc = could not find container \"28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21\": container with ID starting with 28252700aa366c87ee40fdeb6bc98ee0e2365f449c09341bef4ead9eab20ea21 not found: ID does not exist" Sep 18 17:19:14.569442 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.569407 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8"] Sep 18 17:19:14.571640 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.571608 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-21436-predictor-8574dbfbf7-mdtp8"] Sep 18 17:19:14.915881 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:14.915842 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" path="/var/lib/kubelet/pods/30a227ef-983f-44fe-95ec-94656ce728f5/volumes" Sep 18 17:19:17.314355 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:17.314320 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:19:18.538896 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:18.538870 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:19:18.539518 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:18.539488 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:18.916365 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:18.916343 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:19:18.918350 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:18.918327 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:19:18.929863 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:18.929841 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:19:18.932129 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:18.932096 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:19:28.540206 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:28.540156 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:38.540193 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:38.540146 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:41.296333 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.296295 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v"] Sep 18 17:19:41.296754 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.296617 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" containerID="cri-o://ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a" gracePeriod=30 Sep 18 17:19:41.296754 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.296637 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kube-rbac-proxy" containerID="cri-o://bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6" gracePeriod=30 Sep 18 17:19:41.618445 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618403 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9"] Sep 18 17:19:41.618728 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618715 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" Sep 18 17:19:41.618776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618730 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" Sep 18 17:19:41.618776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618747 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kube-rbac-proxy" Sep 18 17:19:41.618776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618753 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kube-rbac-proxy" Sep 18 17:19:41.618878 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618807 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kserve-container" Sep 18 17:19:41.618878 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.618816 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a227ef-983f-44fe-95ec-94656ce728f5" containerName="kube-rbac-proxy" Sep 18 17:19:41.621589 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.621533 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.624227 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.624193 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-792b5-predictor-serving-cert\"" Sep 18 17:19:41.624572 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.624532 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-792b5-kube-rbac-proxy-sar-config\"" Sep 18 17:19:41.632877 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.632849 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9"] Sep 18 17:19:41.647351 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.647311 2586 generic.go:356] "Generic (PLEG): container finished" podID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerID="bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6" exitCode=2 Sep 18 17:19:41.647584 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.647376 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" event={"ID":"792a1fde-2c89-425b-83f2-42d73a9c9317","Type":"ContainerDied","Data":"bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6"} Sep 18 17:19:41.736054 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.736012 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.736254 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.736064 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb9940db-4b7a-4e13-b987-1c560c5a458d-proxy-tls\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.736254 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.736096 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdq4k\" (UniqueName: \"kubernetes.io/projected/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-api-access-rdq4k\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.837595 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.837534 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.837595 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.837599 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb9940db-4b7a-4e13-b987-1c560c5a458d-proxy-tls\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.837830 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.837621 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdq4k\" (UniqueName: \"kubernetes.io/projected/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-api-access-rdq4k\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.838307 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.838287 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.840218 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.840194 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb9940db-4b7a-4e13-b987-1c560c5a458d-proxy-tls\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.848150 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.848117 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdq4k\" (UniqueName: \"kubernetes.io/projected/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-api-access-rdq4k\") pod \"error-404-isvc-792b5-predictor-7bddc6686c-n5rw9\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:41.932299 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:41.932193 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:42.215890 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.215819 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9"] Sep 18 17:19:42.218709 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:19:42.218672 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9940db_4b7a_4e13_b987_1c560c5a458d.slice/crio-2f36ab4902a6ce8b99dd04f3f20786c7ea356888da19591e7062214cc41066e4 WatchSource:0}: Error finding container 2f36ab4902a6ce8b99dd04f3f20786c7ea356888da19591e7062214cc41066e4: Status 404 returned error can't find the container with id 2f36ab4902a6ce8b99dd04f3f20786c7ea356888da19591e7062214cc41066e4 Sep 18 17:19:42.308108 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.308074 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.24:8643/healthz\": dial tcp 10.134.0.24:8643: connect: connection refused" Sep 18 17:19:42.653858 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.653816 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" event={"ID":"bb9940db-4b7a-4e13-b987-1c560c5a458d","Type":"ContainerStarted","Data":"13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f"} Sep 18 17:19:42.654029 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.653901 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" event={"ID":"bb9940db-4b7a-4e13-b987-1c560c5a458d","Type":"ContainerStarted","Data":"379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0"} Sep 18 17:19:42.654029 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.653922 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" event={"ID":"bb9940db-4b7a-4e13-b987-1c560c5a458d","Type":"ContainerStarted","Data":"2f36ab4902a6ce8b99dd04f3f20786c7ea356888da19591e7062214cc41066e4"} Sep 18 17:19:42.654029 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.653945 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:42.675963 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:42.675909 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podStartSLOduration=1.6758930699999999 podStartE2EDuration="1.67589307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:19:42.674751984 +0000 UTC m=+1825.950877186" watchObservedRunningTime="2026-09-18 17:19:42.67589307 +0000 UTC m=+1825.952018242" Sep 18 17:19:43.658572 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:43.658525 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:43.659924 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:43.659894 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:19:44.662192 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:44.662149 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:19:45.344201 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.344163 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:19:45.470469 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.470366 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/792a1fde-2c89-425b-83f2-42d73a9c9317-proxy-tls\") pod \"792a1fde-2c89-425b-83f2-42d73a9c9317\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " Sep 18 17:19:45.470469 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.470414 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlgvg\" (UniqueName: \"kubernetes.io/projected/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-api-access-qlgvg\") pod \"792a1fde-2c89-425b-83f2-42d73a9c9317\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " Sep 18 17:19:45.470469 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.470442 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-rbac-proxy-sar-config\") pod \"792a1fde-2c89-425b-83f2-42d73a9c9317\" (UID: \"792a1fde-2c89-425b-83f2-42d73a9c9317\") " Sep 18 17:19:45.470850 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.470824 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "792a1fde-2c89-425b-83f2-42d73a9c9317" (UID: "792a1fde-2c89-425b-83f2-42d73a9c9317"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:19:45.472860 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.472830 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/792a1fde-2c89-425b-83f2-42d73a9c9317-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "792a1fde-2c89-425b-83f2-42d73a9c9317" (UID: "792a1fde-2c89-425b-83f2-42d73a9c9317"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:19:45.472955 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.472886 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-api-access-qlgvg" (OuterVolumeSpecName: "kube-api-access-qlgvg") pod "792a1fde-2c89-425b-83f2-42d73a9c9317" (UID: "792a1fde-2c89-425b-83f2-42d73a9c9317"). InnerVolumeSpecName "kube-api-access-qlgvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:19:45.571312 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.571252 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/792a1fde-2c89-425b-83f2-42d73a9c9317-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:19:45.571312 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.571305 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlgvg\" (UniqueName: \"kubernetes.io/projected/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-api-access-qlgvg\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:19:45.571312 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.571314 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/792a1fde-2c89-425b-83f2-42d73a9c9317-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:19:45.667479 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.667440 2586 generic.go:356] "Generic (PLEG): container finished" podID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerID="ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a" exitCode=0 Sep 18 17:19:45.667996 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.667490 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" Sep 18 17:19:45.667996 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.667488 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" event={"ID":"792a1fde-2c89-425b-83f2-42d73a9c9317","Type":"ContainerDied","Data":"ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a"} Sep 18 17:19:45.667996 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.667579 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v" event={"ID":"792a1fde-2c89-425b-83f2-42d73a9c9317","Type":"ContainerDied","Data":"30104fc10b8bb8069c14f0a44bab9bc324b55ced48dcd68e1a81adbfa9eb6be3"} Sep 18 17:19:45.667996 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.667607 2586 scope.go:117] "RemoveContainer" containerID="bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6" Sep 18 17:19:45.677128 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.677020 2586 scope.go:117] "RemoveContainer" containerID="ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a" Sep 18 17:19:45.686338 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.686317 2586 scope.go:117] "RemoveContainer" containerID="bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6" Sep 18 17:19:45.686689 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:19:45.686668 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6\": container with ID starting with bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6 not found: ID does not exist" containerID="bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6" Sep 18 17:19:45.686744 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.686699 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6"} err="failed to get container status \"bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6\": rpc error: code = NotFound desc = could not find container \"bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6\": container with ID starting with bc51f6338bb162f7bddeed7d8d442fdc0011840767b601e8243e69f9b22fe3e6 not found: ID does not exist" Sep 18 17:19:45.686744 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.686719 2586 scope.go:117] "RemoveContainer" containerID="ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a" Sep 18 17:19:45.687033 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:19:45.687014 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a\": container with ID starting with ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a not found: ID does not exist" containerID="ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a" Sep 18 17:19:45.687092 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.687040 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a"} err="failed to get container status \"ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a\": rpc error: code = NotFound desc = could not find container \"ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a\": container with ID starting with ed5229933eceaffc9242ba668597ec0583c2ce7866f51add5d0ce9f587c8af5a not found: ID does not exist" Sep 18 17:19:45.690767 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.690741 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v"] Sep 18 17:19:45.694852 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:45.694830 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-78e6b-predictor-547866bf4f-ss64v"] Sep 18 17:19:46.915468 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:46.915428 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" path="/var/lib/kubelet/pods/792a1fde-2c89-425b-83f2-42d73a9c9317/volumes" Sep 18 17:19:48.540252 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:48.540211 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:49.666583 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:49.666524 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:19:49.667087 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:49.667060 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:19:58.540333 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:58.540291 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:19:59.667447 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:19:59.667403 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:20:08.540736 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:20:08.540701 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:20:09.667852 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:20:09.667815 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:20:19.667601 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:20:19.667539 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:20:29.667567 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:20:29.667473 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:20:39.668612 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:20:39.668548 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:24:18.939426 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:24:18.939397 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:24:18.941286 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:24:18.941268 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:24:18.956147 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:24:18.956126 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:24:18.957887 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:24:18.957869 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:28:35.749081 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.749047 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh"] Sep 18 17:28:35.751627 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.749316 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" containerID="cri-o://d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86" gracePeriod=30 Sep 18 17:28:35.751627 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.749413 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kube-rbac-proxy" containerID="cri-o://9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95" gracePeriod=30 Sep 18 17:28:35.984602 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984546 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn"] Sep 18 17:28:35.984899 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984886 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" Sep 18 17:28:35.984944 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984901 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" Sep 18 17:28:35.984944 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984921 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kube-rbac-proxy" Sep 18 17:28:35.984944 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984927 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kube-rbac-proxy" Sep 18 17:28:35.985038 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984978 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kserve-container" Sep 18 17:28:35.985038 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.984985 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="792a1fde-2c89-425b-83f2-42d73a9c9317" containerName="kube-rbac-proxy" Sep 18 17:28:35.987877 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.987857 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:35.992061 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.992034 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-054ab-predictor-serving-cert\"" Sep 18 17:28:35.992385 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:35.992358 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-054ab-kube-rbac-proxy-sar-config\"" Sep 18 17:28:36.006492 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.006407 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn"] Sep 18 17:28:36.142762 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.142724 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.142762 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.142768 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.142982 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.142890 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwhhh\" (UniqueName: \"kubernetes.io/projected/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-api-access-rwhhh\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.243384 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.243344 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwhhh\" (UniqueName: \"kubernetes.io/projected/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-api-access-rwhhh\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.243636 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.243401 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.243636 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.243430 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.243636 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:28:36.243597 2586 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-054ab-predictor-serving-cert: secret "error-404-isvc-054ab-predictor-serving-cert" not found Sep 18 17:28:36.243824 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:28:36.243678 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls podName:bd146766-527c-4b2e-ab51-c9a635f7af64 nodeName:}" failed. No retries permitted until 2026-09-18 17:28:36.743656465 +0000 UTC m=+2360.019781631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls") pod "error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" (UID: "bd146766-527c-4b2e-ab51-c9a635f7af64") : secret "error-404-isvc-054ab-predictor-serving-cert" not found Sep 18 17:28:36.244223 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.244196 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.253907 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.253881 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwhhh\" (UniqueName: \"kubernetes.io/projected/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-api-access-rwhhh\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.726531 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.726497 2586 generic.go:356] "Generic (PLEG): container finished" podID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerID="9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95" exitCode=2 Sep 18 17:28:36.726724 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.726582 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" event={"ID":"df76c216-e80b-41f0-bc1b-1384dc69cb83","Type":"ContainerDied","Data":"9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95"} Sep 18 17:28:36.746961 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.746923 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.749782 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.749754 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls\") pod \"error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:36.898414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:36.898356 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:37.180166 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:37.180133 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn"] Sep 18 17:28:37.182123 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:28:37.182089 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd146766_527c_4b2e_ab51_c9a635f7af64.slice/crio-d844f96d727988f2265b819cd7c43b7bbe67c51166904c6844c2d1cd36ec3048 WatchSource:0}: Error finding container d844f96d727988f2265b819cd7c43b7bbe67c51166904c6844c2d1cd36ec3048: Status 404 returned error can't find the container with id d844f96d727988f2265b819cd7c43b7bbe67c51166904c6844c2d1cd36ec3048 Sep 18 17:28:37.733870 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:37.733830 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" event={"ID":"bd146766-527c-4b2e-ab51-c9a635f7af64","Type":"ContainerStarted","Data":"6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850"} Sep 18 17:28:37.734068 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:37.733898 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" event={"ID":"bd146766-527c-4b2e-ab51-c9a635f7af64","Type":"ContainerStarted","Data":"f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb"} Sep 18 17:28:37.734068 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:37.733928 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" event={"ID":"bd146766-527c-4b2e-ab51-c9a635f7af64","Type":"ContainerStarted","Data":"d844f96d727988f2265b819cd7c43b7bbe67c51166904c6844c2d1cd36ec3048"} Sep 18 17:28:37.734068 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:37.733951 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:37.763828 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:37.763770 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podStartSLOduration=2.763749115 podStartE2EDuration="2.763749115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:28:37.763024066 +0000 UTC m=+2361.039149258" watchObservedRunningTime="2026-09-18 17:28:37.763749115 +0000 UTC m=+2361.039874290" Sep 18 17:28:38.535795 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:38.535741 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused" Sep 18 17:28:38.540232 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:38.540195 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused" Sep 18 17:28:38.737767 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:38.737719 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:38.738989 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:38.738965 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:28:39.302879 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.302849 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:28:39.475351 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.475244 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df76c216-e80b-41f0-bc1b-1384dc69cb83-proxy-tls\") pod \"df76c216-e80b-41f0-bc1b-1384dc69cb83\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " Sep 18 17:28:39.475351 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.475352 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-rbac-proxy-sar-config\") pod \"df76c216-e80b-41f0-bc1b-1384dc69cb83\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " Sep 18 17:28:39.475613 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.475374 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5v8s\" (UniqueName: \"kubernetes.io/projected/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-api-access-t5v8s\") pod \"df76c216-e80b-41f0-bc1b-1384dc69cb83\" (UID: \"df76c216-e80b-41f0-bc1b-1384dc69cb83\") " Sep 18 17:28:39.475776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.475746 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "df76c216-e80b-41f0-bc1b-1384dc69cb83" (UID: "df76c216-e80b-41f0-bc1b-1384dc69cb83"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:28:39.477717 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.477684 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-api-access-t5v8s" (OuterVolumeSpecName: "kube-api-access-t5v8s") pod "df76c216-e80b-41f0-bc1b-1384dc69cb83" (UID: "df76c216-e80b-41f0-bc1b-1384dc69cb83"). InnerVolumeSpecName "kube-api-access-t5v8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:28:39.477820 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.477730 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df76c216-e80b-41f0-bc1b-1384dc69cb83-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "df76c216-e80b-41f0-bc1b-1384dc69cb83" (UID: "df76c216-e80b-41f0-bc1b-1384dc69cb83"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:28:39.576378 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.576340 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:28:39.576378 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.576372 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5v8s\" (UniqueName: \"kubernetes.io/projected/df76c216-e80b-41f0-bc1b-1384dc69cb83-kube-api-access-t5v8s\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:28:39.576378 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.576383 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/df76c216-e80b-41f0-bc1b-1384dc69cb83-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:28:39.743367 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.743279 2586 generic.go:356] "Generic (PLEG): container finished" podID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerID="d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86" exitCode=0 Sep 18 17:28:39.743367 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.743333 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" Sep 18 17:28:39.743590 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.743378 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" event={"ID":"df76c216-e80b-41f0-bc1b-1384dc69cb83","Type":"ContainerDied","Data":"d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86"} Sep 18 17:28:39.743590 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.743442 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh" event={"ID":"df76c216-e80b-41f0-bc1b-1384dc69cb83","Type":"ContainerDied","Data":"e751e8a23f22c68f9d1fcdc316946acd3c143beede8db13ea87f68d2dcc51927"} Sep 18 17:28:39.743590 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.743469 2586 scope.go:117] "RemoveContainer" containerID="9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95" Sep 18 17:28:39.744106 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.744077 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:28:39.754114 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.754071 2586 scope.go:117] "RemoveContainer" containerID="d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86" Sep 18 17:28:39.766241 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.766214 2586 scope.go:117] "RemoveContainer" containerID="9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95" Sep 18 17:28:39.766597 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:28:39.766572 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95\": container with ID starting with 9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95 not found: ID does not exist" containerID="9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95" Sep 18 17:28:39.766667 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.766607 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95"} err="failed to get container status \"9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95\": rpc error: code = NotFound desc = could not find container \"9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95\": container with ID starting with 9cbf0df0bf857934e045fa18433f42eedb83c7533d17ae08035553c2cb634f95 not found: ID does not exist" Sep 18 17:28:39.766667 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.766626 2586 scope.go:117] "RemoveContainer" containerID="d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86" Sep 18 17:28:39.766931 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:28:39.766908 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86\": container with ID starting with d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86 not found: ID does not exist" containerID="d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86" Sep 18 17:28:39.766984 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.766939 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86"} err="failed to get container status \"d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86\": rpc error: code = NotFound desc = could not find container \"d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86\": container with ID starting with d2645938b82d4dbf132552baf6b10640d819ed18cfb9850442c1dd8595ac3d86 not found: ID does not exist" Sep 18 17:28:39.773089 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.773059 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh"] Sep 18 17:28:39.778939 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:39.778911 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-cca0b-predictor-6f7448c4bc-mtmnh"] Sep 18 17:28:40.915529 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:40.915486 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" path="/var/lib/kubelet/pods/df76c216-e80b-41f0-bc1b-1384dc69cb83/volumes" Sep 18 17:28:44.749309 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:44.749278 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:28:44.749885 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:44.749860 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:28:54.750077 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:28:54.750024 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:29:04.750615 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:04.750547 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:29:06.809153 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:06.809116 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9"] Sep 18 17:29:06.809838 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:06.809401 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" containerID="cri-o://379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0" gracePeriod=30 Sep 18 17:29:06.809838 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:06.809434 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kube-rbac-proxy" containerID="cri-o://13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f" gracePeriod=30 Sep 18 17:29:07.150644 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.150606 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg"] Sep 18 17:29:07.151020 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.151002 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" Sep 18 17:29:07.151020 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.151020 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" Sep 18 17:29:07.151174 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.151030 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kube-rbac-proxy" Sep 18 17:29:07.151174 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.151036 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kube-rbac-proxy" Sep 18 17:29:07.151174 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.151085 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kserve-container" Sep 18 17:29:07.151174 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.151093 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="df76c216-e80b-41f0-bc1b-1384dc69cb83" containerName="kube-rbac-proxy" Sep 18 17:29:07.154283 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.154251 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.160635 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.160607 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0cb05-predictor-serving-cert\"" Sep 18 17:29:07.160635 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.160629 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-0cb05-kube-rbac-proxy-sar-config\"" Sep 18 17:29:07.181683 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.181642 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34b2cab2-d82a-48d1-9723-0f3d85195af8-proxy-tls\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.181881 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.181716 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd7k2\" (UniqueName: \"kubernetes.io/projected/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-api-access-kd7k2\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.181881 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.181787 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.187617 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.187549 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg"] Sep 18 17:29:07.283016 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.282961 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.283016 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.283013 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34b2cab2-d82a-48d1-9723-0f3d85195af8-proxy-tls\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.283304 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.283076 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd7k2\" (UniqueName: \"kubernetes.io/projected/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-api-access-kd7k2\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.283753 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.283728 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.285754 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.285731 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34b2cab2-d82a-48d1-9723-0f3d85195af8-proxy-tls\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.302984 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.302943 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd7k2\" (UniqueName: \"kubernetes.io/projected/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-api-access-kd7k2\") pod \"error-404-isvc-0cb05-predictor-c8d4bc477-xvszg\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.465038 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.464938 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:07.742672 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.742640 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg"] Sep 18 17:29:07.745831 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:29:07.745799 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b2cab2_d82a_48d1_9723_0f3d85195af8.slice/crio-84d5dfeb6eda767db8a98a213dd0a2b93d53099f511862ebe254fd9bba5b8901 WatchSource:0}: Error finding container 84d5dfeb6eda767db8a98a213dd0a2b93d53099f511862ebe254fd9bba5b8901: Status 404 returned error can't find the container with id 84d5dfeb6eda767db8a98a213dd0a2b93d53099f511862ebe254fd9bba5b8901 Sep 18 17:29:07.866151 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.866119 2586 generic.go:356] "Generic (PLEG): container finished" podID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerID="13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f" exitCode=2 Sep 18 17:29:07.866548 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.866188 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" event={"ID":"bb9940db-4b7a-4e13-b987-1c560c5a458d","Type":"ContainerDied","Data":"13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f"} Sep 18 17:29:07.867861 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.867831 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" event={"ID":"34b2cab2-d82a-48d1-9723-0f3d85195af8","Type":"ContainerStarted","Data":"05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed"} Sep 18 17:29:07.867985 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:07.867886 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" event={"ID":"34b2cab2-d82a-48d1-9723-0f3d85195af8","Type":"ContainerStarted","Data":"84d5dfeb6eda767db8a98a213dd0a2b93d53099f511862ebe254fd9bba5b8901"} Sep 18 17:29:08.873916 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:08.873877 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" event={"ID":"34b2cab2-d82a-48d1-9723-0f3d85195af8","Type":"ContainerStarted","Data":"7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5"} Sep 18 17:29:08.874302 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:08.874114 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:08.874302 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:08.874156 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:08.875603 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:08.875547 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:08.909957 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:08.909896 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podStartSLOduration=1.9098778589999998 podStartE2EDuration="1.909877859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:29:08.899742855 +0000 UTC m=+2392.175868028" watchObservedRunningTime="2026-09-18 17:29:08.909877859 +0000 UTC m=+2392.186003107" Sep 18 17:29:09.663366 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:09.663312 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused" Sep 18 17:29:09.667839 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:09.667803 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Sep 18 17:29:09.879267 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:09.879230 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:10.660575 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.660522 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:29:10.711775 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.711687 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdq4k\" (UniqueName: \"kubernetes.io/projected/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-api-access-rdq4k\") pod \"bb9940db-4b7a-4e13-b987-1c560c5a458d\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " Sep 18 17:29:10.711775 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.711754 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-rbac-proxy-sar-config\") pod \"bb9940db-4b7a-4e13-b987-1c560c5a458d\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " Sep 18 17:29:10.711775 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.711776 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb9940db-4b7a-4e13-b987-1c560c5a458d-proxy-tls\") pod \"bb9940db-4b7a-4e13-b987-1c560c5a458d\" (UID: \"bb9940db-4b7a-4e13-b987-1c560c5a458d\") " Sep 18 17:29:10.712173 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.712132 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bb9940db-4b7a-4e13-b987-1c560c5a458d" (UID: "bb9940db-4b7a-4e13-b987-1c560c5a458d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:29:10.714037 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.714004 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-api-access-rdq4k" (OuterVolumeSpecName: "kube-api-access-rdq4k") pod "bb9940db-4b7a-4e13-b987-1c560c5a458d" (UID: "bb9940db-4b7a-4e13-b987-1c560c5a458d"). InnerVolumeSpecName "kube-api-access-rdq4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:29:10.714160 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.714082 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb9940db-4b7a-4e13-b987-1c560c5a458d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bb9940db-4b7a-4e13-b987-1c560c5a458d" (UID: "bb9940db-4b7a-4e13-b987-1c560c5a458d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:29:10.812972 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.812916 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:29:10.812972 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.812965 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb9940db-4b7a-4e13-b987-1c560c5a458d-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:29:10.812972 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.812975 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdq4k\" (UniqueName: \"kubernetes.io/projected/bb9940db-4b7a-4e13-b987-1c560c5a458d-kube-api-access-rdq4k\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:29:10.885096 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.885059 2586 generic.go:356] "Generic (PLEG): container finished" podID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerID="379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0" exitCode=0 Sep 18 17:29:10.885508 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.885117 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" Sep 18 17:29:10.885508 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.885151 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" event={"ID":"bb9940db-4b7a-4e13-b987-1c560c5a458d","Type":"ContainerDied","Data":"379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0"} Sep 18 17:29:10.885508 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.885224 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9" event={"ID":"bb9940db-4b7a-4e13-b987-1c560c5a458d","Type":"ContainerDied","Data":"2f36ab4902a6ce8b99dd04f3f20786c7ea356888da19591e7062214cc41066e4"} Sep 18 17:29:10.885508 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.885262 2586 scope.go:117] "RemoveContainer" containerID="13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f" Sep 18 17:29:10.894289 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.894265 2586 scope.go:117] "RemoveContainer" containerID="379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0" Sep 18 17:29:10.902196 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.902174 2586 scope.go:117] "RemoveContainer" containerID="13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f" Sep 18 17:29:10.902547 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:29:10.902527 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f\": container with ID starting with 13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f not found: ID does not exist" containerID="13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f" Sep 18 17:29:10.902651 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.902573 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f"} err="failed to get container status \"13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f\": rpc error: code = NotFound desc = could not find container \"13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f\": container with ID starting with 13032d193a97b86d5834a8c2a3b82f2602cc07cb9ae759d98458745d6a366d6f not found: ID does not exist" Sep 18 17:29:10.902651 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.902591 2586 scope.go:117] "RemoveContainer" containerID="379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0" Sep 18 17:29:10.902904 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:29:10.902882 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0\": container with ID starting with 379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0 not found: ID does not exist" containerID="379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0" Sep 18 17:29:10.902961 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.902909 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0"} err="failed to get container status \"379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0\": rpc error: code = NotFound desc = could not find container \"379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0\": container with ID starting with 379b74797fd532d3e655f69014170bb92c6c8bdd10c7e572d5aab74165b35cc0 not found: ID does not exist" Sep 18 17:29:10.920511 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.920480 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9"] Sep 18 17:29:10.924129 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:10.924101 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-792b5-predictor-7bddc6686c-n5rw9"] Sep 18 17:29:12.915543 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:12.915507 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" path="/var/lib/kubelet/pods/bb9940db-4b7a-4e13-b987-1c560c5a458d/volumes" Sep 18 17:29:14.750386 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:14.750345 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:29:14.884175 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:14.884139 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:29:14.884639 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:14.884613 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:18.970609 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:18.970581 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:29:18.972753 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:18.972733 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:29:18.984212 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:18.984188 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:29:18.986066 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:18.986042 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:29:24.750500 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:24.750447 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Sep 18 17:29:24.885088 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:24.885048 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:34.750794 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:34.750715 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:29:34.884620 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:34.884577 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:44.885344 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:44.885302 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:54.884780 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:54.884741 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Sep 18 17:29:56.930741 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:56.930709 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn"] Sep 18 17:29:56.931163 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:56.930937 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" containerID="cri-o://f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb" gracePeriod=30 Sep 18 17:29:56.931163 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:56.930991 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kube-rbac-proxy" containerID="cri-o://6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850" gracePeriod=30 Sep 18 17:29:57.071593 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.071536 2586 generic.go:356] "Generic (PLEG): container finished" podID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerID="6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850" exitCode=2 Sep 18 17:29:57.071773 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.071596 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" event={"ID":"bd146766-527c-4b2e-ab51-c9a635f7af64","Type":"ContainerDied","Data":"6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850"} Sep 18 17:29:57.159325 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159287 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk"] Sep 18 17:29:57.159741 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159722 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" Sep 18 17:29:57.159741 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159742 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" Sep 18 17:29:57.159914 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159762 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kube-rbac-proxy" Sep 18 17:29:57.159914 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159767 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kube-rbac-proxy" Sep 18 17:29:57.159914 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159816 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kube-rbac-proxy" Sep 18 17:29:57.159914 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.159827 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb9940db-4b7a-4e13-b987-1c560c5a458d" containerName="kserve-container" Sep 18 17:29:57.162925 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.162904 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.167839 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.167814 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0fb0-kube-rbac-proxy-sar-config\"" Sep 18 17:29:57.168203 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.168178 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-f0fb0-predictor-serving-cert\"" Sep 18 17:29:57.174698 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.174670 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk"] Sep 18 17:29:57.186603 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.186499 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.186778 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.186618 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4x8t\" (UniqueName: \"kubernetes.io/projected/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-api-access-q4x8t\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.186778 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.186672 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-proxy-tls\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.287637 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.287582 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-proxy-tls\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.287637 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.287642 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.287865 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.287693 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4x8t\" (UniqueName: \"kubernetes.io/projected/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-api-access-q4x8t\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.288415 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.288388 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.290241 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.290213 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-proxy-tls\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.296737 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.296713 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4x8t\" (UniqueName: \"kubernetes.io/projected/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-api-access-q4x8t\") pod \"error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.473478 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.473391 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:57.745833 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:57.745802 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk"] Sep 18 17:29:57.749239 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:29:57.749212 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b2eb5c_6ea0_42cb_a67e_38592004d3fc.slice/crio-053271273e18509e835a6014a13f4321d559c9a575a07b253026f75ef91cdbe7 WatchSource:0}: Error finding container 053271273e18509e835a6014a13f4321d559c9a575a07b253026f75ef91cdbe7: Status 404 returned error can't find the container with id 053271273e18509e835a6014a13f4321d559c9a575a07b253026f75ef91cdbe7 Sep 18 17:29:58.078929 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.078840 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" event={"ID":"97b2eb5c-6ea0-42cb-a67e-38592004d3fc","Type":"ContainerStarted","Data":"da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa"} Sep 18 17:29:58.078929 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.078905 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" event={"ID":"97b2eb5c-6ea0-42cb-a67e-38592004d3fc","Type":"ContainerStarted","Data":"59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2"} Sep 18 17:29:58.079359 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.078934 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:58.079359 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.078964 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" event={"ID":"97b2eb5c-6ea0-42cb-a67e-38592004d3fc","Type":"ContainerStarted","Data":"053271273e18509e835a6014a13f4321d559c9a575a07b253026f75ef91cdbe7"} Sep 18 17:29:58.079359 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.078993 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:29:58.080485 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.080456 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:29:58.098160 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:58.098111 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podStartSLOduration=1.098094913 podStartE2EDuration="1.098094913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 17:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:29:58.0970366 +0000 UTC m=+2441.373161798" watchObservedRunningTime="2026-09-18 17:29:58.098094913 +0000 UTC m=+2441.374220086" Sep 18 17:29:59.083016 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:59.082981 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:29:59.745018 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:29:59.744959 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused" Sep 18 17:30:00.781839 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.781806 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:30:00.810612 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.810502 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-rbac-proxy-sar-config\") pod \"bd146766-527c-4b2e-ab51-c9a635f7af64\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " Sep 18 17:30:00.810790 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.810643 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwhhh\" (UniqueName: \"kubernetes.io/projected/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-api-access-rwhhh\") pod \"bd146766-527c-4b2e-ab51-c9a635f7af64\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " Sep 18 17:30:00.810790 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.810699 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls\") pod \"bd146766-527c-4b2e-ab51-c9a635f7af64\" (UID: \"bd146766-527c-4b2e-ab51-c9a635f7af64\") " Sep 18 17:30:00.810964 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.810936 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bd146766-527c-4b2e-ab51-c9a635f7af64" (UID: "bd146766-527c-4b2e-ab51-c9a635f7af64"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:30:00.812922 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.812891 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bd146766-527c-4b2e-ab51-c9a635f7af64" (UID: "bd146766-527c-4b2e-ab51-c9a635f7af64"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:30:00.813059 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.812992 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-api-access-rwhhh" (OuterVolumeSpecName: "kube-api-access-rwhhh") pod "bd146766-527c-4b2e-ab51-c9a635f7af64" (UID: "bd146766-527c-4b2e-ab51-c9a635f7af64"). InnerVolumeSpecName "kube-api-access-rwhhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:30:00.911240 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.911202 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:30:00.911240 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.911234 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwhhh\" (UniqueName: \"kubernetes.io/projected/bd146766-527c-4b2e-ab51-c9a635f7af64-kube-api-access-rwhhh\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:30:00.911240 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:00.911244 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd146766-527c-4b2e-ab51-c9a635f7af64-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:30:01.092712 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.092673 2586 generic.go:356] "Generic (PLEG): container finished" podID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerID="f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb" exitCode=0 Sep 18 17:30:01.092918 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.092749 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" Sep 18 17:30:01.092918 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.092765 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" event={"ID":"bd146766-527c-4b2e-ab51-c9a635f7af64","Type":"ContainerDied","Data":"f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb"} Sep 18 17:30:01.092918 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.092858 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn" event={"ID":"bd146766-527c-4b2e-ab51-c9a635f7af64","Type":"ContainerDied","Data":"d844f96d727988f2265b819cd7c43b7bbe67c51166904c6844c2d1cd36ec3048"} Sep 18 17:30:01.092918 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.092887 2586 scope.go:117] "RemoveContainer" containerID="6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850" Sep 18 17:30:01.100931 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.100904 2586 scope.go:117] "RemoveContainer" containerID="f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb" Sep 18 17:30:01.108575 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.108544 2586 scope.go:117] "RemoveContainer" containerID="6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850" Sep 18 17:30:01.108855 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:30:01.108836 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850\": container with ID starting with 6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850 not found: ID does not exist" containerID="6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850" Sep 18 17:30:01.108906 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.108863 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850"} err="failed to get container status \"6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850\": rpc error: code = NotFound desc = could not find container \"6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850\": container with ID starting with 6732eae787d4e81608e55df91431c4ac4f90d212f4ef8689cc2a618b50b1b850 not found: ID does not exist" Sep 18 17:30:01.108906 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.108878 2586 scope.go:117] "RemoveContainer" containerID="f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb" Sep 18 17:30:01.109114 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:30:01.109099 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb\": container with ID starting with f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb not found: ID does not exist" containerID="f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb" Sep 18 17:30:01.109157 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.109120 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb"} err="failed to get container status \"f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb\": rpc error: code = NotFound desc = could not find container \"f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb\": container with ID starting with f331b5f9d79ff182cb4bc28b7212b46453e9dc641628faa722d39c4b5bcd8efb not found: ID does not exist" Sep 18 17:30:01.123545 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.123519 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn"] Sep 18 17:30:01.129499 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:01.129476 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-054ab-predictor-9dbb95cd9-dmxmn"] Sep 18 17:30:02.915798 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:02.915762 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" path="/var/lib/kubelet/pods/bd146766-527c-4b2e-ab51-c9a635f7af64/volumes" Sep 18 17:30:04.088045 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:04.088015 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:30:04.088511 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:04.088484 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:30:04.886571 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:04.886530 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:30:14.088893 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:14.088853 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:30:24.088690 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:24.088645 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:30:34.089015 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:34.088972 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:30:44.089076 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:44.089031 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:30:54.090417 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:30:54.090376 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:34:18.999143 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:34:18.999104 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:34:19.001184 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:34:19.001157 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:34:19.010742 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:34:19.010716 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:34:19.012567 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:34:19.012530 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:39:19.025342 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:19.025316 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:39:19.028181 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:19.028096 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:39:19.039441 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:19.039418 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:39:19.041160 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:19.041143 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:39:22.098481 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:22.098435 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk"] Sep 18 17:39:22.099010 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:22.098703 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" containerID="cri-o://59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2" gracePeriod=30 Sep 18 17:39:22.099010 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:22.098742 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kube-rbac-proxy" containerID="cri-o://da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa" gracePeriod=30 Sep 18 17:39:22.281490 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:22.281455 2586 generic.go:356] "Generic (PLEG): container finished" podID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerID="da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa" exitCode=2 Sep 18 17:39:22.281686 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:22.281519 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" event={"ID":"97b2eb5c-6ea0-42cb-a67e-38592004d3fc","Type":"ContainerDied","Data":"da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa"} Sep 18 17:39:24.083382 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:24.083335 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused" Sep 18 17:39:24.088869 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:24.088837 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Sep 18 17:39:25.549532 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.549497 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:39:25.647427 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.647336 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-proxy-tls\") pod \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " Sep 18 17:39:25.647427 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.647410 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-rbac-proxy-sar-config\") pod \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " Sep 18 17:39:25.647714 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.647443 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4x8t\" (UniqueName: \"kubernetes.io/projected/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-api-access-q4x8t\") pod \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\" (UID: \"97b2eb5c-6ea0-42cb-a67e-38592004d3fc\") " Sep 18 17:39:25.647861 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.647827 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "97b2eb5c-6ea0-42cb-a67e-38592004d3fc" (UID: "97b2eb5c-6ea0-42cb-a67e-38592004d3fc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:39:25.649727 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.649705 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-api-access-q4x8t" (OuterVolumeSpecName: "kube-api-access-q4x8t") pod "97b2eb5c-6ea0-42cb-a67e-38592004d3fc" (UID: "97b2eb5c-6ea0-42cb-a67e-38592004d3fc"). InnerVolumeSpecName "kube-api-access-q4x8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:39:25.649811 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.649747 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "97b2eb5c-6ea0-42cb-a67e-38592004d3fc" (UID: "97b2eb5c-6ea0-42cb-a67e-38592004d3fc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:39:25.748197 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.748143 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:39:25.748197 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.748187 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:39:25.748197 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:25.748197 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4x8t\" (UniqueName: \"kubernetes.io/projected/97b2eb5c-6ea0-42cb-a67e-38592004d3fc-kube-api-access-q4x8t\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:39:26.297033 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.296994 2586 generic.go:356] "Generic (PLEG): container finished" podID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerID="59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2" exitCode=0 Sep 18 17:39:26.297200 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.297040 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" event={"ID":"97b2eb5c-6ea0-42cb-a67e-38592004d3fc","Type":"ContainerDied","Data":"59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2"} Sep 18 17:39:26.297200 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.297062 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" Sep 18 17:39:26.297200 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.297090 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk" event={"ID":"97b2eb5c-6ea0-42cb-a67e-38592004d3fc","Type":"ContainerDied","Data":"053271273e18509e835a6014a13f4321d559c9a575a07b253026f75ef91cdbe7"} Sep 18 17:39:26.297200 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.297116 2586 scope.go:117] "RemoveContainer" containerID="da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa" Sep 18 17:39:26.305924 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.305907 2586 scope.go:117] "RemoveContainer" containerID="59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2" Sep 18 17:39:26.312823 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.312804 2586 scope.go:117] "RemoveContainer" containerID="da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa" Sep 18 17:39:26.313093 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:39:26.313072 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa\": container with ID starting with da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa not found: ID does not exist" containerID="da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa" Sep 18 17:39:26.313164 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.313105 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa"} err="failed to get container status \"da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa\": rpc error: code = NotFound desc = could not find container \"da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa\": container with ID starting with da2fb7044cc34c1adef9c436b94fffc853f6a660c57143a6aaa33922d18d01aa not found: ID does not exist" Sep 18 17:39:26.313164 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.313129 2586 scope.go:117] "RemoveContainer" containerID="59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2" Sep 18 17:39:26.313395 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:39:26.313378 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2\": container with ID starting with 59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2 not found: ID does not exist" containerID="59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2" Sep 18 17:39:26.313434 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.313402 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2"} err="failed to get container status \"59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2\": rpc error: code = NotFound desc = could not find container \"59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2\": container with ID starting with 59364b1f2c28b95ae196b9aeb87e0bd5bedd59f45d6470c4ebfc91afbc2964e2 not found: ID does not exist" Sep 18 17:39:26.320185 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.320164 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk"] Sep 18 17:39:26.323789 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.323771 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-f0fb0-predictor-668cf4bb89-qq7hk"] Sep 18 17:39:26.915523 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:39:26.915493 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" path="/var/lib/kubelet/pods/97b2eb5c-6ea0-42cb-a67e-38592004d3fc/volumes" Sep 18 17:44:19.056827 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:44:19.056720 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:44:19.060804 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:44:19.058506 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:44:19.061053 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:44:19.061036 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:44:19.063138 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:44:19.063118 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:46:37.192323 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:37.192288 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg"] Sep 18 17:46:37.192885 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:37.192525 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" containerID="cri-o://05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed" gracePeriod=30 Sep 18 17:46:37.192885 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:37.192633 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kube-rbac-proxy" containerID="cri-o://7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5" gracePeriod=30 Sep 18 17:46:37.944874 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:37.944829 2586 generic.go:356] "Generic (PLEG): container finished" podID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerID="7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5" exitCode=2 Sep 18 17:46:37.945052 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:37.944895 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" event={"ID":"34b2cab2-d82a-48d1-9723-0f3d85195af8","Type":"ContainerDied","Data":"7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5"} Sep 18 17:46:39.879569 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:39.879525 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused" Sep 18 17:46:40.844065 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.844034 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:46:40.930414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.930318 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-rbac-proxy-sar-config\") pod \"34b2cab2-d82a-48d1-9723-0f3d85195af8\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " Sep 18 17:46:40.930414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.930376 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34b2cab2-d82a-48d1-9723-0f3d85195af8-proxy-tls\") pod \"34b2cab2-d82a-48d1-9723-0f3d85195af8\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " Sep 18 17:46:40.930414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.930408 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd7k2\" (UniqueName: \"kubernetes.io/projected/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-api-access-kd7k2\") pod \"34b2cab2-d82a-48d1-9723-0f3d85195af8\" (UID: \"34b2cab2-d82a-48d1-9723-0f3d85195af8\") " Sep 18 17:46:40.930880 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.930748 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "34b2cab2-d82a-48d1-9723-0f3d85195af8" (UID: "34b2cab2-d82a-48d1-9723-0f3d85195af8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 18 17:46:40.932746 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.932723 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-api-access-kd7k2" (OuterVolumeSpecName: "kube-api-access-kd7k2") pod "34b2cab2-d82a-48d1-9723-0f3d85195af8" (UID: "34b2cab2-d82a-48d1-9723-0f3d85195af8"). InnerVolumeSpecName "kube-api-access-kd7k2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 18 17:46:40.932933 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.932910 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34b2cab2-d82a-48d1-9723-0f3d85195af8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "34b2cab2-d82a-48d1-9723-0f3d85195af8" (UID: "34b2cab2-d82a-48d1-9723-0f3d85195af8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 18 17:46:40.959006 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.958968 2586 generic.go:356] "Generic (PLEG): container finished" podID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerID="05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed" exitCode=0 Sep 18 17:46:40.959192 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.959022 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" Sep 18 17:46:40.959192 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.959051 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" event={"ID":"34b2cab2-d82a-48d1-9723-0f3d85195af8","Type":"ContainerDied","Data":"05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed"} Sep 18 17:46:40.959192 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.959120 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg" event={"ID":"34b2cab2-d82a-48d1-9723-0f3d85195af8","Type":"ContainerDied","Data":"84d5dfeb6eda767db8a98a213dd0a2b93d53099f511862ebe254fd9bba5b8901"} Sep 18 17:46:40.959192 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.959155 2586 scope.go:117] "RemoveContainer" containerID="7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5" Sep 18 17:46:40.968156 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.968131 2586 scope.go:117] "RemoveContainer" containerID="05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed" Sep 18 17:46:40.976241 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.976219 2586 scope.go:117] "RemoveContainer" containerID="7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5" Sep 18 17:46:40.976624 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:46:40.976600 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5\": container with ID starting with 7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5 not found: ID does not exist" containerID="7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5" Sep 18 17:46:40.976685 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.976640 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5"} err="failed to get container status \"7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5\": rpc error: code = NotFound desc = could not find container \"7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5\": container with ID starting with 7157be8467f150b2f83ad8c77b7e1d9f622559621b6b54eef10e507bea825ec5 not found: ID does not exist" Sep 18 17:46:40.976685 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.976666 2586 scope.go:117] "RemoveContainer" containerID="05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed" Sep 18 17:46:40.976988 ip-10-0-135-188 kubenswrapper[2586]: E0918 17:46:40.976968 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed\": container with ID starting with 05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed not found: ID does not exist" containerID="05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed" Sep 18 17:46:40.977033 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.976994 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed"} err="failed to get container status \"05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed\": rpc error: code = NotFound desc = could not find container \"05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed\": container with ID starting with 05cf8cbee0c6881f095a288d23ea3905ad44f5f6015e0d6c7791f499aa8e34ed not found: ID does not exist" Sep 18 17:46:40.982225 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.982191 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg"] Sep 18 17:46:40.986490 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:40.986459 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-0cb05-predictor-c8d4bc477-xvszg"] Sep 18 17:46:41.031771 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:41.031729 2586 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:46:41.031771 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:41.031762 2586 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/34b2cab2-d82a-48d1-9723-0f3d85195af8-proxy-tls\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:46:41.031771 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:41.031773 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd7k2\" (UniqueName: \"kubernetes.io/projected/34b2cab2-d82a-48d1-9723-0f3d85195af8-kube-api-access-kd7k2\") on node \"ip-10-0-135-188.ec2.internal\" DevicePath \"\"" Sep 18 17:46:42.916767 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:46:42.916718 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" path="/var/lib/kubelet/pods/34b2cab2-d82a-48d1-9723-0f3d85195af8/volumes" Sep 18 17:47:12.726065 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:12.726033 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9hg2q_b2aace0d-8144-4803-b013-63eeaf617aca/global-pull-secret-syncer/0.log" Sep 18 17:47:13.009232 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:13.009139 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tbjtj_be06cb5e-12f9-4bf7-b5ce-eb68f9864842/konnectivity-agent/0.log" Sep 18 17:47:13.060340 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:13.060307 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-188.ec2.internal_99da832300d4ddecffd299754cc04b87/haproxy/0.log" Sep 18 17:47:18.711664 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:18.711623 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-954n4_f7707d9e-e512-4252-b07a-e26a44bf885c/node-exporter/0.log" Sep 18 17:47:18.757264 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:18.757228 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-954n4_f7707d9e-e512-4252-b07a-e26a44bf885c/kube-rbac-proxy/0.log" Sep 18 17:47:18.801810 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:18.801731 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-954n4_f7707d9e-e512-4252-b07a-e26a44bf885c/init-textfile/0.log" Sep 18 17:47:20.562752 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.562717 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg"] Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.562970 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.562980 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.562993 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.562998 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563004 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563010 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563017 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563022 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563033 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563037 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563045 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563051 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563091 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563099 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563107 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563113 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="34b2cab2-d82a-48d1-9723-0f3d85195af8" containerName="kube-rbac-proxy" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563119 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd146766-527c-4b2e-ab51-c9a635f7af64" containerName="kserve-container" Sep 18 17:47:20.563297 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.563125 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b2eb5c-6ea0-42cb-a67e-38592004d3fc" containerName="kube-rbac-proxy" Sep 18 17:47:20.565767 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.565750 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.570620 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.570592 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9l6m9\"/\"default-dockercfg-drjkb\"" Sep 18 17:47:20.570620 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.570613 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9l6m9\"/\"openshift-service-ca.crt\"" Sep 18 17:47:20.570849 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.570596 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9l6m9\"/\"kube-root-ca.crt\"" Sep 18 17:47:20.583701 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.583661 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg"] Sep 18 17:47:20.736087 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.736047 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-proc\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.736264 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.736092 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-podres\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.736264 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.736182 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scw8h\" (UniqueName: \"kubernetes.io/projected/ff3bd949-53b3-4512-b83a-12c428090b16-kube-api-access-scw8h\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.736264 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.736228 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-lib-modules\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.736385 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.736252 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-sys\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837044 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837016 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-proc\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837132 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837057 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-podres\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837182 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837153 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-proc\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837219 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837187 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scw8h\" (UniqueName: \"kubernetes.io/projected/ff3bd949-53b3-4512-b83a-12c428090b16-kube-api-access-scw8h\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837219 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837173 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-podres\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837285 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837242 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-lib-modules\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837285 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837281 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-sys\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837351 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837335 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-lib-modules\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.837388 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.837357 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff3bd949-53b3-4512-b83a-12c428090b16-sys\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.848972 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.848947 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scw8h\" (UniqueName: \"kubernetes.io/projected/ff3bd949-53b3-4512-b83a-12c428090b16-kube-api-access-scw8h\") pod \"perf-node-gather-daemonset-8czgg\" (UID: \"ff3bd949-53b3-4512-b83a-12c428090b16\") " pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:20.874876 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:20.874837 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:21.151181 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:21.151094 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg"] Sep 18 17:47:21.154604 ip-10-0-135-188 kubenswrapper[2586]: W0918 17:47:21.154569 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff3bd949_53b3_4512_b83a_12c428090b16.slice/crio-21f22ac40f950b72ed05c2ef210a0df875ef32f8b0ee0a35668bebd3a9e01e1e WatchSource:0}: Error finding container 21f22ac40f950b72ed05c2ef210a0df875ef32f8b0ee0a35668bebd3a9e01e1e: Status 404 returned error can't find the container with id 21f22ac40f950b72ed05c2ef210a0df875ef32f8b0ee0a35668bebd3a9e01e1e Sep 18 17:47:21.845627 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:21.845593 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b4fc4cb9c-fwr7b_27c8f7d0-221c-4284-a9b7-fe602797e70d/console/0.log" Sep 18 17:47:22.113071 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:22.112983 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" event={"ID":"ff3bd949-53b3-4512-b83a-12c428090b16","Type":"ContainerStarted","Data":"8cb6693f207bb9a0cf606cc0c5ee64d3ee938737fef3aa799a0998ddf3e825ce"} Sep 18 17:47:22.113233 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:22.113075 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" event={"ID":"ff3bd949-53b3-4512-b83a-12c428090b16","Type":"ContainerStarted","Data":"21f22ac40f950b72ed05c2ef210a0df875ef32f8b0ee0a35668bebd3a9e01e1e"} Sep 18 17:47:22.113233 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:22.113133 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:22.154776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:22.154713 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" podStartSLOduration=2.15469634 podStartE2EDuration="2.15469634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 17:47:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 17:47:22.15175926 +0000 UTC m=+3485.427884447" watchObservedRunningTime="2026-09-18 17:47:22.15469634 +0000 UTC m=+3485.430821513" Sep 18 17:47:23.393699 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:23.393673 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dn64f_4196f06f-d002-4987-a299-3e65a0e22677/dns/0.log" Sep 18 17:47:23.420625 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:23.420589 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dn64f_4196f06f-d002-4987-a299-3e65a0e22677/kube-rbac-proxy/0.log" Sep 18 17:47:23.600857 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:23.600826 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vvmcg_dad9b91f-014d-4fdc-a1f3-838a7df5196d/dns-node-resolver/0.log" Sep 18 17:47:24.128847 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:24.128810 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4x25g_871e64c9-5bf6-4560-aac6-6403d5f56df6/node-ca/0.log" Sep 18 17:47:25.350250 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:25.350224 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b2m4k_066f325b-02fb-440a-bd43-8efdf244f004/serve-healthcheck-canary/0.log" Sep 18 17:47:25.803776 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:25.803684 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b64vj_043c6094-46a5-46ea-998d-964d015ed767/kube-rbac-proxy/0.log" Sep 18 17:47:25.834321 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:25.834285 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b64vj_043c6094-46a5-46ea-998d-964d015ed767/exporter/0.log" Sep 18 17:47:25.860591 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:25.860539 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b64vj_043c6094-46a5-46ea-998d-964d015ed767/extractor/0.log" Sep 18 17:47:28.126899 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:28.126874 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9l6m9/perf-node-gather-daemonset-8czgg" Sep 18 17:47:28.435597 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:28.435486 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-cd9dp_c8ddcf41-3ab3-4802-9c7d-3647a7abada9/manager/0.log" Sep 18 17:47:34.784685 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:34.784652 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/3.log" Sep 18 17:47:34.796627 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:34.796598 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-285nx_1ccfd74a-5cdf-48ff-af9c-f1f56c5e55f3/kube-multus/4.log" Sep 18 17:47:35.209322 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.209294 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/kube-multus-additional-cni-plugins/0.log" Sep 18 17:47:35.233707 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.233671 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/egress-router-binary-copy/0.log" Sep 18 17:47:35.257785 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.257762 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/cni-plugins/0.log" Sep 18 17:47:35.282209 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.282185 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/bond-cni-plugin/0.log" Sep 18 17:47:35.309045 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.309012 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/routeoverride-cni/0.log" Sep 18 17:47:35.332972 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.332939 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/whereabouts-cni-bincopy/0.log" Sep 18 17:47:35.357661 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.357630 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8kdgf_581ad636-7b18-43bc-803c-8c163b820c0f/whereabouts-cni/0.log" Sep 18 17:47:35.455358 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.455324 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5jqgt_776f65d4-9fed-4694-8139-fb8bb9a95c7b/network-metrics-daemon/0.log" Sep 18 17:47:35.478017 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:35.477935 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5jqgt_776f65d4-9fed-4694-8139-fb8bb9a95c7b/kube-rbac-proxy/0.log" Sep 18 17:47:36.362840 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.362797 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovn-controller/0.log" Sep 18 17:47:36.382440 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.382389 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovn-acl-logging/0.log" Sep 18 17:47:36.408448 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.408416 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/kube-rbac-proxy-node/0.log" Sep 18 17:47:36.435272 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.435242 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/kube-rbac-proxy-ovn-metrics/0.log" Sep 18 17:47:36.461340 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.461312 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/northd/0.log" Sep 18 17:47:36.484905 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.484869 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/nbdb/0.log" Sep 18 17:47:36.510135 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.510101 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/sbdb/0.log" Sep 18 17:47:36.560724 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.560692 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/0.log" Sep 18 17:47:36.588414 ip-10-0-135-188 kubenswrapper[2586]: I0918 17:47:36.588385 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8hzb9_1297f04c-ff7b-4c98-a6ee-9fa6b887a561/ovnkube-controller/1.log"