Jul 08 18:15:46.083856 ip-10-0-131-106 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 18:15:46.083871 ip-10-0-131-106 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 18:15:46.083880 ip-10-0-131-106 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 18:15:46.084335 ip-10-0-131-106 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 18:15:56.153109 ip-10-0-131-106 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 18:15:56.153135 ip-10-0-131-106 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 54f35887a6e743668fba701e9cb7f37f -- Jul 08 18:18:05.422366 ip-10-0-131-106 systemd[1]: Starting Kubernetes Kubelet... Jul 08 18:18:05.835062 ip-10-0-131-106 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:18:05.835062 ip-10-0-131-106 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 18:18:05.835062 ip-10-0-131-106 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:18:05.835062 ip-10-0-131-106 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 18:18:05.835062 ip-10-0-131-106 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 18:18:05.837538 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.837445 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 18:18:05.844239 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844212 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:18:05.844239 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844235 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:18:05.844239 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844241 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:18:05.844239 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844244 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:18:05.844239 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844248 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844251 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844255 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844258 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844261 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844264 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844266 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844271 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844275 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844277 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844280 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844282 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844285 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844287 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844290 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844292 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844295 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844297 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844300 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:18:05.844444 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844302 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844305 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844308 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844310 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844313 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844316 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844318 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844321 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844323 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844325 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844330 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844334 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844337 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844340 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844342 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844346 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844349 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844351 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844354 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844357 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:18:05.844894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844359 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844361 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844364 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844366 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844369 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844371 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844373 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844376 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844378 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844381 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844384 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844386 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844388 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844391 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844393 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844396 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844398 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844400 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844403 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844405 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:18:05.845434 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844408 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844412 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844414 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844417 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844419 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844421 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844424 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844427 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844429 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844432 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844434 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844437 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844439 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844442 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844445 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844447 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844450 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844453 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844455 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844458 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:18:05.845912 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844461 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844464 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844466 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844469 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844471 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844474 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844477 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844479 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844481 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844484 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844486 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844488 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844491 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844493 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844497 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.844500 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845210 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845218 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845221 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845225 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:18:05.846416 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845228 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845231 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845234 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845236 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845238 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845241 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845245 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845248 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845251 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845253 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845256 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845258 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845261 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845264 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845266 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845269 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845271 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845274 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845276 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845279 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:18:05.846898 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845281 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845283 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845286 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845288 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845291 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845294 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845296 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845299 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845301 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845303 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845306 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845311 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845314 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845316 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845319 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845321 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845324 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845327 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845330 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845332 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:18:05.847414 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845334 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845337 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845339 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845342 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845344 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845347 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845350 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845354 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845357 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845360 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845364 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845366 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845369 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845373 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845376 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845379 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845382 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845384 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845386 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845389 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:18:05.847894 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845391 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845394 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845396 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845399 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845402 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845406 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845409 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845412 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845415 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845417 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845420 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845423 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845425 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845428 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845431 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845434 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845436 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845439 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845442 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:18:05.848410 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845445 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845448 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845450 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845453 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845455 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845458 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845460 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845463 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845465 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845468 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845470 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845473 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845476 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845478 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845480 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845483 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845861 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845865 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845868 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845871 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:18:05.848878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845873 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845876 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845879 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845881 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845884 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845886 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845889 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845891 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845894 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845896 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845898 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845901 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845905 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845907 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845911 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845913 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845916 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845918 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845922 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:18:05.849413 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845924 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845927 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845929 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845932 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845935 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845937 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845939 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845942 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845944 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845947 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845950 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845953 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845955 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845957 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845960 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845962 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845965 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845968 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845970 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845972 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:18:05.849874 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845975 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845977 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845981 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845985 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.845988 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846006 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846010 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846013 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846016 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846018 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846021 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846024 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846026 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846029 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846031 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846033 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846036 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846038 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846041 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846043 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:18:05.850365 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846046 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846051 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846054 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846058 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846061 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846064 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846067 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846069 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846072 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846075 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846077 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846080 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846083 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846085 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846088 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846091 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846094 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846096 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846099 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:18:05.850888 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846104 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846108 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846110 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846113 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846115 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846118 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846120 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846123 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846125 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846128 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846130 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846132 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846135 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846137 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846141 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846144 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846147 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846219 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846227 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846234 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846238 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 18:18:05.851495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846242 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846245 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846250 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846254 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846257 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846259 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846263 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846266 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846269 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846272 2562 flags.go:64] FLAG: --cgroup-root="" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846274 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846277 2562 flags.go:64] FLAG: --client-ca-file="" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846281 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846284 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846288 2562 flags.go:64] FLAG: --cluster-domain="" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846291 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846293 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846298 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846302 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846305 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846308 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846311 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846314 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846317 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 08 18:18:05.852009 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846320 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846324 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846327 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846330 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846334 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846337 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846340 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846343 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846345 2562 flags.go:64] FLAG: --enable-server="true" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846348 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846353 2562 flags.go:64] FLAG: --event-burst="100" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846356 2562 flags.go:64] FLAG: --event-qps="50" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846359 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846362 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846364 2562 flags.go:64] FLAG: --eviction-hard="" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846368 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846371 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846374 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846376 2562 flags.go:64] FLAG: --eviction-soft="" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846379 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846382 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846385 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846388 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846391 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846394 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 18:18:05.852629 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846398 2562 flags.go:64] FLAG: --feature-gates="" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846402 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846405 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846408 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846411 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846413 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846416 2562 flags.go:64] FLAG: --help="false" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846419 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-131-106.ec2.internal" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846424 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846427 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846430 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846433 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846436 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846439 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846443 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846453 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846456 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846459 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846462 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846464 2562 flags.go:64] FLAG: --kube-reserved="" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846467 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846470 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846473 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846476 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 18:18:05.853269 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846479 2562 flags.go:64] FLAG: --lock-file="" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846481 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846484 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846487 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846492 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846496 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846499 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846502 2562 flags.go:64] FLAG: --logging-format="text" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846506 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846509 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846512 2562 flags.go:64] FLAG: --manifest-url="" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846514 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846518 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846521 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846525 2562 flags.go:64] FLAG: --max-pods="110" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846528 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846531 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846534 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846537 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846539 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846542 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846545 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846552 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846555 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846558 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 18:18:05.853915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846561 2562 flags.go:64] FLAG: --pod-cidr="" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846563 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846568 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846571 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846574 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846576 2562 flags.go:64] FLAG: --port="10250" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846579 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846582 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-052c570ed5a246012" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846585 2562 flags.go:64] FLAG: --qos-reserved="" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846588 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846590 2562 flags.go:64] FLAG: --register-node="true" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846593 2562 flags.go:64] FLAG: --register-with-taints="" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846596 2562 flags.go:64] FLAG: --registry-burst="10" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846600 2562 flags.go:64] FLAG: --registry-qps="5" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846602 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846611 2562 flags.go:64] FLAG: --reserved-memory="" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846614 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846617 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846620 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846622 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846625 2562 flags.go:64] FLAG: --runonce="false" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846628 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846630 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846633 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846636 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 18:18:05.854559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846639 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846642 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846645 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846648 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846651 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846653 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846656 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846659 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846662 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846664 2562 flags.go:64] FLAG: --system-cgroups="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846667 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846672 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846675 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846677 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846681 2562 flags.go:64] FLAG: --tls-min-version="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846684 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846686 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846689 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846692 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846694 2562 flags.go:64] FLAG: --v="2" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846698 2562 flags.go:64] FLAG: --version="false" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846703 2562 flags.go:64] FLAG: --vmodule="" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846707 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.846710 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846834 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:18:05.855184 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846838 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846841 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846844 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846847 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846849 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846852 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846854 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846857 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846859 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846862 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846866 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846869 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846872 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846875 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846878 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846882 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846885 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846887 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846890 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:18:05.855820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846892 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846895 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846897 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846900 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846902 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846905 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846907 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846910 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846912 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846915 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846923 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846925 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846928 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846930 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846933 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846935 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846938 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846940 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846943 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846945 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:18:05.856306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846948 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846950 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846952 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846955 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846957 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846959 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846962 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846964 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846967 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846969 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846972 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846974 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846977 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846979 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846981 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846984 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846986 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.846988 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847004 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847007 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:18:05.856788 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847010 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847012 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847016 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847019 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847022 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847025 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847027 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847030 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847032 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847035 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847037 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847041 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847043 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847046 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847048 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847051 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847053 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847056 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847058 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847061 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:18:05.857293 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847064 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847066 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847069 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847071 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847074 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847076 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847079 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847081 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847084 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847086 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847088 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847091 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847094 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847097 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847102 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847105 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847115 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847118 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:18:05.857782 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.847121 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.848089 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855078 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855095 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855116 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855122 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855176 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855181 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855185 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855188 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855191 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855194 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855198 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855201 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855204 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:18:05.858232 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855207 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855210 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855213 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855216 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855219 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855222 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855225 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855227 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855230 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855232 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855235 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855237 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855242 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855245 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855248 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855251 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855255 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855257 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855260 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:18:05.858663 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855263 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855267 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855270 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855272 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855275 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855277 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855279 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855282 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855284 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855287 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855289 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855291 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855294 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855296 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855298 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855301 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855303 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855305 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855308 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855310 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:18:05.859159 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855313 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855315 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855317 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855319 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855322 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855325 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855327 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855330 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855332 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855335 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855337 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855340 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855342 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855346 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855349 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855352 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855354 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855356 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855359 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855361 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:18:05.859647 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855363 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855366 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855369 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855372 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855375 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855378 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855381 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855384 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855386 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855389 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855391 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855394 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855396 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855399 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855401 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855403 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855406 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855410 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855414 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:18:05.860151 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855417 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855419 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855422 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855424 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855427 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855429 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855432 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855434 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855437 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855440 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855442 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855444 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855450 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855578 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 18:18:05.860613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855584 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855586 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855589 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855592 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855595 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855597 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855600 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855602 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855605 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855607 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855610 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855613 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855615 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855618 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855620 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855622 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855625 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855627 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855631 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855635 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 18:18:05.860964 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855637 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855640 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855642 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855645 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855647 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855650 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855652 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855655 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855657 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855660 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855662 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855665 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855667 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855670 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855672 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855674 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855677 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855679 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855682 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855684 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 18:18:05.861485 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855687 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855689 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855691 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855694 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855696 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855699 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855701 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855703 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855706 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855709 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855712 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855714 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855716 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855719 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855721 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855724 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855726 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855729 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855731 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 18:18:05.861968 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855733 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855736 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855739 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855741 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855743 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855746 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855748 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855750 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855753 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855756 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855758 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855761 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855763 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855766 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855768 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855771 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855773 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855775 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855778 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855780 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 18:18:05.862440 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855782 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855785 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855789 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855792 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855795 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855797 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855800 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855804 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855807 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855810 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855813 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855816 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855819 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855822 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855825 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855827 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855830 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855832 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 18:18:05.862950 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:05.855835 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855840 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.855970 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.858199 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.859544 2562 server.go:1019] "Starting client certificate rotation" Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.859638 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.859685 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 18:18:05.863403 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.862945 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:18:05.867566 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.867540 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 18:18:05.880513 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.880491 2562 log.go:25] "Validated CRI v1 runtime API" Jul 08 18:18:05.882867 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.882850 2562 log.go:25] "Validated CRI v1 image API" Jul 08 18:18:05.883686 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.883670 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 18:18:05.887256 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.887232 2562 fs.go:135] Filesystem UUIDs: map[12230607-094c-4198-9e8f-5ec2bdd46e35:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9134e19d-21fd-43c0-bbfa-6f19164ec7b0:/dev/nvme0n1p3] Jul 08 18:18:05.887314 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.887257 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 18:18:05.891793 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.891777 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:18:05.892465 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.892353 2562 manager.go:216] Machine: {Timestamp:2026-07-08 18:18:05.890925475 +0000 UTC m=+0.347259787 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099531 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2df12d5d54222d406e10761e123001 SystemUUID:ec2df12d-5d54-222d-406e-10761e123001 BootID:54f35887-a6e7-4366-8fba-701e9cb7f37f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:04:d5:3f:fb:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:04:d5:3f:fb:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:0e:3b:f6:54:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 18:18:05.892465 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.892462 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 18:18:05.892557 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.892548 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 18:18:05.896032 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.896005 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 18:18:05.896174 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.896036 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-106.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 18:18:05.897281 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.897271 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 18:18:05.897319 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.897285 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 18:18:05.897319 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.897293 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:18:05.897319 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.897306 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 18:18:05.898751 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.898741 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 18:18:05.901847 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.901836 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:18:05.901970 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.901960 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 18:18:05.902016 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902006 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 08 18:18:05.902016 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902016 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 18:18:05.902074 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902036 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 18:18:05.902074 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902044 2562 kubelet.go:398] "Adding apiserver pod source" Jul 08 18:18:05.902074 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902053 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 18:18:05.902439 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902426 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:18:05.902496 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.902460 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 18:18:05.904867 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.904839 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 18:18:05.907082 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.907065 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 18:18:05.907166 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.907105 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 18:18:05.909328 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909299 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 18:18:05.909418 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909385 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 18:18:05.909418 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909397 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 18:18:05.909418 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909406 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 18:18:05.909418 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909420 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909430 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909443 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909452 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909470 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909479 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909492 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 18:18:05.909533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.909501 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 18:18:05.910342 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.910323 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 18:18:05.910426 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.910345 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 18:18:05.913846 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.913829 2562 server.go:1268] "Started kubelet" Jul 08 18:18:05.913960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.913925 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 18:18:05.914028 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.913952 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 18:18:05.914077 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.914036 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 18:18:05.914637 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.914616 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-106.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 18:18:05.914741 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.914682 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 18:18:05.914785 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.914753 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 18:18:05.914909 ip-10-0-131-106 systemd[1]: Started Kubernetes Kubelet. Jul 08 18:18:05.915544 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.915521 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 18:18:05.916199 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.916180 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wr2k" Jul 08 18:18:05.916279 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.916251 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 08 18:18:05.919499 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.919477 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 18:18:05.921564 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.921549 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 18:18:05.921564 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.921557 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 18:18:05.921685 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.921655 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:05.922354 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.922126 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 18:18:05.922913 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.922896 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 18:18:05.922913 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.922911 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 18:18:05.923140 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.923124 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 08 18:18:05.923140 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.923138 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 08 18:18:05.923284 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.923260 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8wr2k" Jul 08 18:18:05.924112 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924062 2562 factory.go:55] Registering systemd factory Jul 08 18:18:05.924112 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924080 2562 factory.go:223] Registration of the systemd container factory successfully Jul 08 18:18:05.924364 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924335 2562 factory.go:153] Registering CRI-O factory Jul 08 18:18:05.924364 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924359 2562 factory.go:223] Registration of the crio container factory successfully Jul 08 18:18:05.924481 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924413 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 18:18:05.924481 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924437 2562 factory.go:103] Registering Raw factory Jul 08 18:18:05.924481 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.924452 2562 manager.go:1196] Started watching for new ooms in manager Jul 08 18:18:05.925294 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.925276 2562 manager.go:349] Starting recovery of all containers Jul 08 18:18:05.925425 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.925359 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 18:18:05.925610 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.925334 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 18:18:05.927132 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.925716 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-106.ec2.internal.18c06358e67adeb0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-106.ec2.internal,UID:ip-10-0-131-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-106.ec2.internal,},FirstTimestamp:2026-07-08 18:18:05.913800368 +0000 UTC m=+0.370134681,LastTimestamp:2026-07-08 18:18:05.913800368 +0000 UTC m=+0.370134681,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-106.ec2.internal,}" Jul 08 18:18:05.936660 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.936493 2562 manager.go:354] Recovery completed Jul 08 18:18:05.941935 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.941921 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:05.944130 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944113 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:05.944244 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944145 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:05.944244 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944156 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:05.944753 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944741 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 18:18:05.944753 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944753 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 18:18:05.944831 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944770 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 08 18:18:05.944831 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.944743 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:05.945505 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.945491 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:05.945575 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.945516 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:05.945575 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.945526 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:05.946119 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.946106 2562 policy_none.go:49] "None policy: Start" Jul 08 18:18:05.946166 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.946126 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 18:18:05.946166 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.946136 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 18:18:05.947261 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.947250 2562 policy_none.go:47] "Start" Jul 08 18:18:05.985122 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985107 2562 manager.go:337] "Starting Device Plugin manager" Jul 08 18:18:05.985225 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.985150 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 18:18:05.985225 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985165 2562 server.go:87] "Starting device plugin registration server" Jul 08 18:18:05.985441 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985427 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 18:18:05.985496 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985442 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 18:18:05.985553 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985515 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 18:18:05.985608 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985596 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 18:18:05.985654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:05.985610 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 18:18:05.986228 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.986206 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 18:18:05.986332 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:05.986255 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:06.086285 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.086218 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:06.087458 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.087196 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:06.087458 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.087226 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:06.087458 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.087240 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:06.087458 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.087266 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.094798 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.094778 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.094908 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.094806 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-106.ec2.internal\": node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:06.105072 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.105047 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:06.125360 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.125330 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 18:18:06.126749 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.126724 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 18:18:06.126826 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.126775 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 18:18:06.126826 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.126801 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 18:18:06.126905 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.126842 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 18:18:06.130679 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.130661 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:06.205735 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.205698 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:06.228020 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.227180 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal"] Jul 08 18:18:06.228098 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.228069 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:06.229074 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.229060 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:06.229142 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.229091 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:06.229142 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.229104 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:06.230461 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.230441 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:06.230586 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.230572 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.230625 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.230602 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:06.231194 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.231178 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:06.231295 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.231210 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:06.231295 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.231211 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:06.231295 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.231228 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:06.231295 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.231244 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:06.231295 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.231260 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:06.232599 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.232579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.232698 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.232612 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 18:18:06.233281 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.233262 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientMemory" Jul 08 18:18:06.233379 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.233294 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 18:18:06.233379 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.233304 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeHasSufficientPID" Jul 08 18:18:06.251599 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.251572 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-106.ec2.internal\" not found" node="ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.255931 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.255915 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-106.ec2.internal\" not found" node="ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.306845 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.306808 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:06.325191 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.325161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dee390bc22bbe9a006e589c9e7a83c0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal\" (UID: \"dee390bc22bbe9a006e589c9e7a83c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.325191 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.325191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/188ef7b029f22b3628e6f2183332d139-config\") pod \"kube-apiserver-proxy-ip-10-0-131-106.ec2.internal\" (UID: \"188ef7b029f22b3628e6f2183332d139\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.325364 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.325210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dee390bc22bbe9a006e589c9e7a83c0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal\" (UID: \"dee390bc22bbe9a006e589c9e7a83c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.407646 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:06.407580 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:06.425950 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.425922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dee390bc22bbe9a006e589c9e7a83c0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal\" (UID: \"dee390bc22bbe9a006e589c9e7a83c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.426054 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.425955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dee390bc22bbe9a006e589c9e7a83c0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal\" (UID: \"dee390bc22bbe9a006e589c9e7a83c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.426054 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.425973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/188ef7b029f22b3628e6f2183332d139-config\") pod \"kube-apiserver-proxy-ip-10-0-131-106.ec2.internal\" (UID: \"188ef7b029f22b3628e6f2183332d139\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.426054 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.426021 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/188ef7b029f22b3628e6f2183332d139-config\") pod \"kube-apiserver-proxy-ip-10-0-131-106.ec2.internal\" (UID: \"188ef7b029f22b3628e6f2183332d139\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.426054 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.426042 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dee390bc22bbe9a006e589c9e7a83c0e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal\" (UID: \"dee390bc22bbe9a006e589c9e7a83c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:06.426190 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:06.426048 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dee390bc22bbe9a006e589c9e7a83c0e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal\" (UID: \"dee390bc22bbe9a006e589c9e7a83c0e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:07.702402 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:07.702352 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:07.748845 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:07.748820 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:07.753937 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:07.753920 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" Jul 08 18:18:07.803415 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:07.803379 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:07.904016 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:07.903925 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:08.004506 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.004475 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-106.ec2.internal\" not found" Jul 08 18:18:08.028241 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.028214 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:08.053251 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.053222 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 18:18:08.053864 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.053417 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:18:08.053864 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.053418 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 18:18:08.096538 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.096499 2562 apiserver.go:52] "Watching apiserver" Jul 08 18:18:08.114054 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.114022 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 18:18:08.115920 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.115902 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 18:18:08.116049 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.116035 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" Jul 08 18:18:08.116442 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.116418 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-v5fds","openshift-multus/network-metrics-daemon-cgqjt","openshift-network-diagnostics/network-check-target-597wz","openshift-ovn-kubernetes/ovnkube-node-jfzsd","kube-system/global-pull-secret-syncer-j2f2w","kube-system/konnectivity-agent-wmnmp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb","openshift-cluster-node-tuning-operator/tuned-rk8vt","openshift-dns/node-resolver-76cqb","openshift-multus/multus-8fqbw","openshift-multus/multus-additional-cni-plugins-mjcct","openshift-network-operator/iptables-alerter-shn8r"] Jul 08 18:18:08.118079 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.118064 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.119693 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.119470 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.119693 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.119612 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:08.121994 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.120808 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.121994 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.121055 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mpbz6\"" Jul 08 18:18:08.121994 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.121234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 18:18:08.121994 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.121454 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.122219 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.122138 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:08.122301 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.122282 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:08.122855 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.122834 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d9n2\" (UniqueName: \"kubernetes.io/projected/8bfb8749-101a-4ec2-a4ee-19f590ad1350-kube-api-access-4d9n2\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.122895 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.122880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8bfb8749-101a-4ec2-a4ee-19f590ad1350-host\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.122927 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.122910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8bfb8749-101a-4ec2-a4ee-19f590ad1350-serviceca\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.123001 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.122960 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 18:13:07 +0000 UTC" deadline="2028-03-03 22:00:55.33225795 +0000 UTC" Jul 08 18:18:08.123033 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.123004 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14499h42m47.209257312s" Jul 08 18:18:08.124153 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.124139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.125335 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.125318 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.125417 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.125396 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:08.126505 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.126487 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.126592 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.126522 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.126592 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.126549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 18:18:08.126691 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.126607 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 18:18:08.126780 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.126761 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.127544 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.127518 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 18:18:08.127544 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.127541 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 18:18:08.127671 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.127608 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:18:08.127726 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.127708 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" Jul 08 18:18:08.127823 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.127808 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7zdck\"" Jul 08 18:18:08.127896 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.127882 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.128756 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.128739 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 18:18:08.128993 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.128980 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cwg49\"" Jul 08 18:18:08.129335 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.129253 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.130176 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.130157 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 18:18:08.130253 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.130179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tszg6\"" Jul 08 18:18:08.130253 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.130232 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 18:18:08.130381 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.130165 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.130381 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.130159 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.130840 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.130823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.131449 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.131429 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.131552 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.131477 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mwshw\"" Jul 08 18:18:08.131552 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.131528 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.132308 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.132289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.132957 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.132933 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 18:18:08.132957 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.132941 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.133138 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.133012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tk92d\"" Jul 08 18:18:08.133439 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.133420 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.133923 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.133908 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.134467 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.134454 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.134674 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.134661 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.134778 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.134760 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 18:18:08.134843 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.134760 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 18:18:08.134843 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.134825 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5jpc2\"" Jul 08 18:18:08.135381 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.135364 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal"] Jul 08 18:18:08.135478 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.135468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.136223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.136207 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 18:18:08.136467 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.136452 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 18:18:08.136582 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.136495 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r6xvx\"" Jul 08 18:18:08.137514 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.137498 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:18:08.137779 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.137766 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 18:18:08.137878 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.137864 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 18:18:08.138017 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.138003 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-f5bks\"" Jul 08 18:18:08.139541 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.139527 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal"] Jul 08 18:18:08.139603 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.139586 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 18:18:08.150737 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.150718 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7gf92" Jul 08 18:18:08.158597 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.158532 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7gf92" Jul 08 18:18:08.218280 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.218241 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 18:18:08.223573 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223550 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-daemon-config\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.223704 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8bfb8749-101a-4ec2-a4ee-19f590ad1350-serviceca\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.223704 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovn-node-metrics-cert\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.223704 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcp68\" (UniqueName: \"kubernetes.io/projected/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-kube-api-access-rcp68\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.223704 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-sys\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.223841 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-socket-dir-parent\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.223841 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87wff\" (UniqueName: \"kubernetes.io/projected/4298d484-fff8-4835-8fdd-d409e875fcaa-kube-api-access-87wff\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.223841 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223822 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.223960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d9n2\" (UniqueName: \"kubernetes.io/projected/8bfb8749-101a-4ec2-a4ee-19f590ad1350-kube-api-access-4d9n2\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.223960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223870 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d774ec5-c014-4267-92bc-9857e91e8dab-cni-binary-copy\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.223960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223887 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-cni-multus\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.223960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.223960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-os-release\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.223976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-systemd\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224000 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-node-log\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-cni-netd\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-lib-modules\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224048 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8bfb8749-101a-4ec2-a4ee-19f590ad1350-serviceca\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-netns\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b59bbc9f-1106-47b1-ae05-4ebc641b1249-dbus\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-device-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224137 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-etc-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsncz\" (UniqueName: \"kubernetes.io/projected/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-kube-api-access-wsncz\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.224223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224205 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224237 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wqf6\" (UniqueName: \"kubernetes.io/projected/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-kube-api-access-6wqf6\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnlpx\" (UniqueName: \"kubernetes.io/projected/31aa72aa-4b52-4112-a90a-f8b9f4784008-kube-api-access-jnlpx\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-kubelet\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-cni-bin\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224332 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-registration-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-sys-fs\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tzgk\" (UniqueName: \"kubernetes.io/projected/7e3da2e7-0256-4012-8b64-129f535b3358-kube-api-access-4tzgk\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-log-socket\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224410 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-var-lib-kubelet\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-kubelet\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-etc-kubernetes\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224452 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-socket-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4298d484-fff8-4835-8fdd-d409e875fcaa-hosts-file\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4298d484-fff8-4835-8fdd-d409e875fcaa-tmp-dir\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cnibin\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.224685 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/acec3e89-3c81-41eb-a93b-72c59f4eb441-agent-certs\") pod \"konnectivity-agent-wmnmp\" (UID: \"acec3e89-3c81-41eb-a93b-72c59f4eb441\") " pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-run-netns\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-cni-bin\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-env-overrides\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk5w2\" (UniqueName: \"kubernetes.io/projected/6d774ec5-c014-4267-92bc-9857e91e8dab-kube-api-access-dk5w2\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6lj\" (UniqueName: \"kubernetes.io/projected/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-kube-api-access-tt6lj\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-ovn\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysctl-d\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-etc-selinux\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-kubernetes\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-systemd\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-tmp\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.224999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-system-cni-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.225452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-k8s-cni-cncf-io\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-iptables-alerter-script\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225095 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-os-release\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-system-cni-dir\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225202 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-var-lib-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-systemd-units\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-slash\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovnkube-config\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-host\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-multus-certs\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysctl-conf\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225363 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-tuned\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b59bbc9f-1106-47b1-ae05-4ebc641b1249-kubelet-config\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.225938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225403 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/acec3e89-3c81-41eb-a93b-72c59f4eb441-konnectivity-ca\") pod \"konnectivity-agent-wmnmp\" (UID: \"acec3e89-3c81-41eb-a93b-72c59f4eb441\") " pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-modprobe-d\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysconfig\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-run\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225485 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-cni-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225512 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-cnibin\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-conf-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8bfb8749-101a-4ec2-a4ee-19f590ad1350-host\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225560 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovnkube-script-lib\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-hostroot\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225616 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8bfb8749-101a-4ec2-a4ee-19f590ad1350-host\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.226429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.225643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-host-slash\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.232165 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.231984 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 18:18:08.235966 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.235906 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d9n2\" (UniqueName: \"kubernetes.io/projected/8bfb8749-101a-4ec2-a4ee-19f590ad1350-kube-api-access-4d9n2\") pod \"node-ca-v5fds\" (UID: \"8bfb8749-101a-4ec2-a4ee-19f590ad1350\") " pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.320589 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.320553 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188ef7b029f22b3628e6f2183332d139.slice/crio-006c79e4d4286c6657798854951bde93902ad0c92bd1ad78ec2d5ec1c11c35d7 WatchSource:0}: Error finding container 006c79e4d4286c6657798854951bde93902ad0c92bd1ad78ec2d5ec1c11c35d7: Status 404 returned error can't find the container with id 006c79e4d4286c6657798854951bde93902ad0c92bd1ad78ec2d5ec1c11c35d7 Jul 08 18:18:08.321127 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.321102 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddee390bc22bbe9a006e589c9e7a83c0e.slice/crio-03c3615b6fdcdfa5f6f8362cd5e2ae55c5f62ff12d5f28f1e4b1c7e2eff51401 WatchSource:0}: Error finding container 03c3615b6fdcdfa5f6f8362cd5e2ae55c5f62ff12d5f28f1e4b1c7e2eff51401: Status 404 returned error can't find the container with id 03c3615b6fdcdfa5f6f8362cd5e2ae55c5f62ff12d5f28f1e4b1c7e2eff51401 Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/acec3e89-3c81-41eb-a93b-72c59f4eb441-konnectivity-ca\") pod \"konnectivity-agent-wmnmp\" (UID: \"acec3e89-3c81-41eb-a93b-72c59f4eb441\") " pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-modprobe-d\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysconfig\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-run\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-cni-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-cnibin\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326361 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-conf-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326474 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-cni-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326507 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-modprobe-d\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326707 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-cnibin\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-conf-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326823 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysconfig\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovnkube-script-lib\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326882 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-run\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-hostroot\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.328290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.326944 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/acec3e89-3c81-41eb-a93b-72c59f4eb441-konnectivity-ca\") pod \"konnectivity-agent-wmnmp\" (UID: \"acec3e89-3c81-41eb-a93b-72c59f4eb441\") " pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-host-slash\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327053 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-hostroot\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-daemon-config\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327153 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-host-slash\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327241 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovn-node-metrics-cert\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327453 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovnkube-script-lib\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcp68\" (UniqueName: \"kubernetes.io/projected/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-kube-api-access-rcp68\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-sys\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-socket-dir-parent\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87wff\" (UniqueName: \"kubernetes.io/projected/4298d484-fff8-4835-8fdd-d409e875fcaa-kube-api-access-87wff\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d774ec5-c014-4267-92bc-9857e91e8dab-cni-binary-copy\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-cni-multus\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327771 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-daemon-config\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327782 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-os-release\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.329479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327812 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-systemd\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-node-log\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.327876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-cni-netd\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-lib-modules\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328126 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328177 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-netns\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cni-binary-copy\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-sys\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d774ec5-c014-4267-92bc-9857e91e8dab-cni-binary-copy\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.328863 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.328939 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:08.828917953 +0000 UTC m=+2.090983709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.328128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-netns\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329019 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-node-log\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329022 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-multus-socket-dir-parent\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329055 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b59bbc9f-1106-47b1-ae05-4ebc641b1249-dbus\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329084 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-os-release\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329081 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-cni-netd\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329099 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-device-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.330201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329232 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-cni-multus\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b59bbc9f-1106-47b1-ae05-4ebc641b1249-dbus\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329311 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-systemd\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329355 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-lib-modules\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329364 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-device-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-etc-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsncz\" (UniqueName: \"kubernetes.io/projected/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-kube-api-access-wsncz\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-etc-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329555 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wqf6\" (UniqueName: \"kubernetes.io/projected/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-kube-api-access-6wqf6\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnlpx\" (UniqueName: \"kubernetes.io/projected/31aa72aa-4b52-4112-a90a-f8b9f4784008-kube-api-access-jnlpx\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329634 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-kubelet\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-cni-bin\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-registration-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-sys-fs\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.331018 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329749 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tzgk\" (UniqueName: \"kubernetes.io/projected/7e3da2e7-0256-4012-8b64-129f535b3358-kube-api-access-4tzgk\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-log-socket\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-var-lib-kubelet\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-kubelet\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-etc-kubernetes\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-socket-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4298d484-fff8-4835-8fdd-d409e875fcaa-hosts-file\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4298d484-fff8-4835-8fdd-d409e875fcaa-tmp-dir\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.329976 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cnibin\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/acec3e89-3c81-41eb-a93b-72c59f4eb441-agent-certs\") pod \"konnectivity-agent-wmnmp\" (UID: \"acec3e89-3c81-41eb-a93b-72c59f4eb441\") " pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-log-socket\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-run-netns\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-cni-bin\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330094 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-env-overrides\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330166 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-etc-kubernetes\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovn-node-metrics-cert\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-var-lib-kubelet\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk5w2\" (UniqueName: \"kubernetes.io/projected/6d774ec5-c014-4267-92bc-9857e91e8dab-kube-api-access-dk5w2\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.331982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330234 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4298d484-fff8-4835-8fdd-d409e875fcaa-hosts-file\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330356 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-cni-bin\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330416 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330422 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-cnibin\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-registration-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330472 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-sys-fs\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330461 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-var-lib-kubelet\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330494 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-cni-bin\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6lj\" (UniqueName: \"kubernetes.io/projected/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-kube-api-access-tt6lj\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-ovn\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330563 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysctl-d\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-etc-selinux\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330591 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-env-overrides\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.332829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-kubelet\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-run-ovn\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330747 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4298d484-fff8-4835-8fdd-d409e875fcaa-tmp-dir\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330751 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-etc-selinux\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330819 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-kubernetes\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-systemd\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330877 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-kubernetes\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330900 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-tmp\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330923 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-systemd\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-system-cni-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330983 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysctl-d\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330985 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-system-cni-dir\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.330985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-k8s-cni-cncf-io\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331077 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331078 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-k8s-cni-cncf-io\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-iptables-alerter-script\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331136 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-os-release\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.333450 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331174 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7e3da2e7-0256-4012-8b64-129f535b3358-socket-dir\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.331132 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.331300 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:08.831279034 +0000 UTC m=+2.093344777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331316 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-os-release\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-system-cni-dir\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331377 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-var-lib-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-systemd-units\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331433 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-system-cni-dir\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-slash\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331458 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331479 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-var-lib-openvswitch\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331492 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovnkube-config\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-systemd-units\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331538 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-host\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.334389 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331558 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-slash\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-host\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331570 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-multus-certs\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331658 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d774ec5-c014-4267-92bc-9857e91e8dab-host-run-multus-certs\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331742 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-host-run-netns\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysctl-conf\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-tuned\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b59bbc9f-1106-47b1-ae05-4ebc641b1249-kubelet-config\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331892 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b59bbc9f-1106-47b1-ae05-4ebc641b1249-kubelet-config\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.331897 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-sysctl-conf\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.332238 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-iptables-alerter-script\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.333202 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/acec3e89-3c81-41eb-a93b-72c59f4eb441-agent-certs\") pod \"konnectivity-agent-wmnmp\" (UID: \"acec3e89-3c81-41eb-a93b-72c59f4eb441\") " pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.333503 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-tmp\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.333632 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-ovnkube-config\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.334998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.334142 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-etc-tuned\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.337428 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.337153 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87wff\" (UniqueName: \"kubernetes.io/projected/4298d484-fff8-4835-8fdd-d409e875fcaa-kube-api-access-87wff\") pod \"node-resolver-76cqb\" (UID: \"4298d484-fff8-4835-8fdd-d409e875fcaa\") " pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.337428 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.337253 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:08.337428 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.337300 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:08.337428 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.337316 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:08.337428 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.337391 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:18:08.837374648 +0000 UTC m=+2.099440403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:08.338019 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.337450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcp68\" (UniqueName: \"kubernetes.io/projected/f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce-kube-api-access-rcp68\") pod \"ovnkube-node-jfzsd\" (UID: \"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.338852 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.338832 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tzgk\" (UniqueName: \"kubernetes.io/projected/7e3da2e7-0256-4012-8b64-129f535b3358-kube-api-access-4tzgk\") pod \"aws-ebs-csi-driver-node-b4wpb\" (UID: \"7e3da2e7-0256-4012-8b64-129f535b3358\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.338934 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.338856 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnlpx\" (UniqueName: \"kubernetes.io/projected/31aa72aa-4b52-4112-a90a-f8b9f4784008-kube-api-access-jnlpx\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.339130 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.339112 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsncz\" (UniqueName: \"kubernetes.io/projected/cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e-kube-api-access-wsncz\") pod \"tuned-rk8vt\" (UID: \"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e\") " pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.339192 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.339168 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6lj\" (UniqueName: \"kubernetes.io/projected/beb08a44-9fdd-42a2-9366-a6bcbeb85cb5-kube-api-access-tt6lj\") pod \"multus-additional-cni-plugins-mjcct\" (UID: \"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5\") " pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.339577 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.339553 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wqf6\" (UniqueName: \"kubernetes.io/projected/1cc41f43-1ed9-4045-923d-a4b5c47cd31a-kube-api-access-6wqf6\") pod \"iptables-alerter-shn8r\" (UID: \"1cc41f43-1ed9-4045-923d-a4b5c47cd31a\") " pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.339880 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.339864 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk5w2\" (UniqueName: \"kubernetes.io/projected/6d774ec5-c014-4267-92bc-9857e91e8dab-kube-api-access-dk5w2\") pod \"multus-8fqbw\" (UID: \"6d774ec5-c014-4267-92bc-9857e91e8dab\") " pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.436480 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.436383 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:08.448655 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.448624 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5fds" Jul 08 18:18:08.455100 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.455067 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" Jul 08 18:18:08.457979 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.457927 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfb8749_101a_4ec2_a4ee_19f590ad1350.slice/crio-90e324b9de9424370fffff7e62a900090be4efce1fb3fcbea59a21a93530bd7e WatchSource:0}: Error finding container 90e324b9de9424370fffff7e62a900090be4efce1fb3fcbea59a21a93530bd7e: Status 404 returned error can't find the container with id 90e324b9de9424370fffff7e62a900090be4efce1fb3fcbea59a21a93530bd7e Jul 08 18:18:08.462525 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.462503 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7d0626_8a75_4e39_adbe_c4e04a9e2e5e.slice/crio-2dbaa96590c6c405bba1da023683354e9b1848f2bbabce0987f224df96877d14 WatchSource:0}: Error finding container 2dbaa96590c6c405bba1da023683354e9b1848f2bbabce0987f224df96877d14: Status 404 returned error can't find the container with id 2dbaa96590c6c405bba1da023683354e9b1848f2bbabce0987f224df96877d14 Jul 08 18:18:08.472710 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.472681 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:08.481873 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.481840 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:08.481993 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.481956 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ed1352_d8a0_4c55_a504_8cf3cc12b2ce.slice/crio-3db05a57d77a886fb850cc59601fdd46e6b18957019b8d101f2da91cda57d6a6 WatchSource:0}: Error finding container 3db05a57d77a886fb850cc59601fdd46e6b18957019b8d101f2da91cda57d6a6: Status 404 returned error can't find the container with id 3db05a57d77a886fb850cc59601fdd46e6b18957019b8d101f2da91cda57d6a6 Jul 08 18:18:08.490220 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.490192 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacec3e89_3c81_41eb_a93b_72c59f4eb441.slice/crio-ff440e09a09dd3a5679c4e8acd09f721d40e8fbad1c386f9e1fc6608f91e38ac WatchSource:0}: Error finding container ff440e09a09dd3a5679c4e8acd09f721d40e8fbad1c386f9e1fc6608f91e38ac: Status 404 returned error can't find the container with id ff440e09a09dd3a5679c4e8acd09f721d40e8fbad1c386f9e1fc6608f91e38ac Jul 08 18:18:08.506433 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.506401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" Jul 08 18:18:08.513576 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.513555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-76cqb" Jul 08 18:18:08.514471 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.514451 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e3da2e7_0256_4012_8b64_129f535b3358.slice/crio-960bd0dd6b83054ac29d87bb1e1ba8ac3e2651a4b115046310455466804d851b WatchSource:0}: Error finding container 960bd0dd6b83054ac29d87bb1e1ba8ac3e2651a4b115046310455466804d851b: Status 404 returned error can't find the container with id 960bd0dd6b83054ac29d87bb1e1ba8ac3e2651a4b115046310455466804d851b Jul 08 18:18:08.521025 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.521002 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4298d484_fff8_4835_8fdd_d409e875fcaa.slice/crio-9ad89607de42f2ca2f03ccd26ba8d5e04e93c7748d573986947a138c613675d4 WatchSource:0}: Error finding container 9ad89607de42f2ca2f03ccd26ba8d5e04e93c7748d573986947a138c613675d4: Status 404 returned error can't find the container with id 9ad89607de42f2ca2f03ccd26ba8d5e04e93c7748d573986947a138c613675d4 Jul 08 18:18:08.557230 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.557199 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8fqbw" Jul 08 18:18:08.562875 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.562857 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mjcct" Jul 08 18:18:08.564316 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.564297 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-shn8r" Jul 08 18:18:08.568198 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.567537 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d774ec5_c014_4267_92bc_9857e91e8dab.slice/crio-637245725b17e57eeaf985670de6dcd5fbaa80e3471fd7932b24070434c754f7 WatchSource:0}: Error finding container 637245725b17e57eeaf985670de6dcd5fbaa80e3471fd7932b24070434c754f7: Status 404 returned error can't find the container with id 637245725b17e57eeaf985670de6dcd5fbaa80e3471fd7932b24070434c754f7 Jul 08 18:18:08.572560 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.572537 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb08a44_9fdd_42a2_9366_a6bcbeb85cb5.slice/crio-870d2c87ada42a66a12f1414891c45dfab2f9546c5a1433e2fa506088a8d9aad WatchSource:0}: Error finding container 870d2c87ada42a66a12f1414891c45dfab2f9546c5a1433e2fa506088a8d9aad: Status 404 returned error can't find the container with id 870d2c87ada42a66a12f1414891c45dfab2f9546c5a1433e2fa506088a8d9aad Jul 08 18:18:08.574808 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:08.574787 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc41f43_1ed9_4045_923d_a4b5c47cd31a.slice/crio-48e993e75fc78cc7bbca6d0d19971a3829463d1eee6a6dae092fb94453b5d94f WatchSource:0}: Error finding container 48e993e75fc78cc7bbca6d0d19971a3829463d1eee6a6dae092fb94453b5d94f: Status 404 returned error can't find the container with id 48e993e75fc78cc7bbca6d0d19971a3829463d1eee6a6dae092fb94453b5d94f Jul 08 18:18:08.585850 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.585830 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:08.835208 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.834447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:08.835208 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.834522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:08.835208 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.834666 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:08.835208 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.834716 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:09.83469911 +0000 UTC m=+3.096764853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:08.835208 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.835101 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:08.835208 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.835141 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:09.83513009 +0000 UTC m=+3.097195821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:08.935076 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:08.935040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:08.935242 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.935200 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:08.935242 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.935221 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:08.935242 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.935234 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:08.935379 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:08.935310 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:18:09.935290042 +0000 UTC m=+3.197355778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:09.161520 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.161363 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:13:08 +0000 UTC" deadline="2028-01-27 15:20:10.535927447 +0000 UTC" Jul 08 18:18:09.161520 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.161422 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13629h2m1.374509286s" Jul 08 18:18:09.245498 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.245469 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:09.324559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.324526 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:09.324726 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.324659 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:09.324785 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.324754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:09.324859 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.324839 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:09.343911 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.343852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" event={"ID":"188ef7b029f22b3628e6f2183332d139","Type":"ContainerStarted","Data":"006c79e4d4286c6657798854951bde93902ad0c92bd1ad78ec2d5ec1c11c35d7"} Jul 08 18:18:09.351006 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.350965 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" event={"ID":"dee390bc22bbe9a006e589c9e7a83c0e","Type":"ContainerStarted","Data":"03c3615b6fdcdfa5f6f8362cd5e2ae55c5f62ff12d5f28f1e4b1c7e2eff51401"} Jul 08 18:18:09.357863 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.357825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shn8r" event={"ID":"1cc41f43-1ed9-4045-923d-a4b5c47cd31a","Type":"ContainerStarted","Data":"48e993e75fc78cc7bbca6d0d19971a3829463d1eee6a6dae092fb94453b5d94f"} Jul 08 18:18:09.363832 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.363799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerStarted","Data":"870d2c87ada42a66a12f1414891c45dfab2f9546c5a1433e2fa506088a8d9aad"} Jul 08 18:18:09.376391 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.376355 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" event={"ID":"7e3da2e7-0256-4012-8b64-129f535b3358","Type":"ContainerStarted","Data":"960bd0dd6b83054ac29d87bb1e1ba8ac3e2651a4b115046310455466804d851b"} Jul 08 18:18:09.381563 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.381531 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8fqbw" event={"ID":"6d774ec5-c014-4267-92bc-9857e91e8dab","Type":"ContainerStarted","Data":"637245725b17e57eeaf985670de6dcd5fbaa80e3471fd7932b24070434c754f7"} Jul 08 18:18:09.396649 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.396588 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-76cqb" event={"ID":"4298d484-fff8-4835-8fdd-d409e875fcaa","Type":"ContainerStarted","Data":"9ad89607de42f2ca2f03ccd26ba8d5e04e93c7748d573986947a138c613675d4"} Jul 08 18:18:09.419676 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.419591 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wmnmp" event={"ID":"acec3e89-3c81-41eb-a93b-72c59f4eb441","Type":"ContainerStarted","Data":"ff440e09a09dd3a5679c4e8acd09f721d40e8fbad1c386f9e1fc6608f91e38ac"} Jul 08 18:18:09.438637 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.438596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"3db05a57d77a886fb850cc59601fdd46e6b18957019b8d101f2da91cda57d6a6"} Jul 08 18:18:09.455841 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.455759 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" event={"ID":"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e","Type":"ContainerStarted","Data":"2dbaa96590c6c405bba1da023683354e9b1848f2bbabce0987f224df96877d14"} Jul 08 18:18:09.457656 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.457598 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5fds" event={"ID":"8bfb8749-101a-4ec2-a4ee-19f590ad1350","Type":"ContainerStarted","Data":"90e324b9de9424370fffff7e62a900090be4efce1fb3fcbea59a21a93530bd7e"} Jul 08 18:18:09.844322 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.844279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:09.844510 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.844370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:09.844510 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.844495 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:09.844612 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.844561 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:11.844541203 +0000 UTC m=+5.106606934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:09.845009 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.844958 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:09.845097 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.845018 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:11.845003511 +0000 UTC m=+5.107069242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:09.946305 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:09.945568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:09.946305 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.945745 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:09.946305 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.945763 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:09.946305 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.945775 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:09.946305 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:09.945890 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:18:11.945870259 +0000 UTC m=+5.207936004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:10.162712 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:10.162540 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:13:08 +0000 UTC" deadline="2028-01-07 13:36:33.145263639 +0000 UTC" Jul 08 18:18:10.162712 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:10.162580 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13147h18m22.98268843s" Jul 08 18:18:10.322292 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:10.322238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:10.322489 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:10.322434 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:11.106336 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:11.106309 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 18:18:11.324191 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:11.324156 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:11.324791 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.324305 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:11.324791 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:11.324707 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:11.324926 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.324811 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:11.862368 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:11.862332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:11.862545 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:11.862402 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:11.862545 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.862487 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:11.862747 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.862556 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:15.862536843 +0000 UTC m=+9.124602596 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:11.862747 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.862578 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:11.862747 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.862656 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:15.862637279 +0000 UTC m=+9.124703019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:11.964479 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:11.963805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:11.964479 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.963993 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:11.964479 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.964011 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:11.964479 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.964024 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:11.964479 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:11.964082 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:18:15.964065387 +0000 UTC m=+9.226131132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:12.321586 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:12.321505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:12.321734 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:12.321660 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:13.324276 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:13.324231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:13.324662 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:13.324380 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:13.324792 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:13.324777 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:13.324887 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:13.324870 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:14.321877 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:14.321842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:14.322064 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:14.321994 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:15.341727 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:15.341468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:15.342131 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:15.341533 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:15.342131 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:15.341766 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:15.342131 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:15.341856 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:15.899442 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:15.899393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:15.899642 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:15.899479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:15.899642 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:15.899563 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:15.899642 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:15.899631 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:23.899612802 +0000 UTC m=+17.161678533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:15.899805 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:15.899645 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:15.899805 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:15.899700 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:23.899686644 +0000 UTC m=+17.161752378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:16.000052 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:16.000015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:16.000229 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:16.000152 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:16.000229 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:16.000166 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:16.000229 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:16.000175 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:16.000229 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:16.000227 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:18:24.000214724 +0000 UTC m=+17.262280471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:16.322878 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:16.322283 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:16.322878 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:16.322450 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:17.322224 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:17.322196 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:17.322684 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:17.322309 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:17.322684 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:17.322424 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:17.322684 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:17.322542 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:18.322157 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:18.321410 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:18.322157 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:18.321543 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:19.322417 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:19.322380 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:19.322417 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:19.322399 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:19.322880 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:19.322521 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:19.322880 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:19.322671 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:20.322239 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:20.322153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:20.322413 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:20.322314 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:21.321704 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:21.321667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:21.322145 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:21.321666 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:21.322145 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:21.321794 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:21.322145 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:21.321859 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:22.321279 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:22.321231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:22.321455 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:22.321363 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:23.324167 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:23.324143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:23.324598 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:23.324248 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:23.324598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:23.324296 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:23.324598 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:23.324414 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:23.964380 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:23.964345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:23.964523 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:23.964392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:23.964523 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:23.964486 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:23.964523 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:23.964488 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:23.964615 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:23.964541 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:39.964527626 +0000 UTC m=+33.226593355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:23.964615 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:23.964555 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:39.964548979 +0000 UTC m=+33.226614709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:24.065687 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:24.065654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:24.065949 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:24.065865 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:24.065949 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:24.065895 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:24.065949 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:24.065909 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:24.066539 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:24.066520 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:18:40.065959356 +0000 UTC m=+33.328025108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:24.322251 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:24.322170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:24.322510 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:24.322320 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:25.321790 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:25.321759 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:25.322291 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:25.321765 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:25.322291 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:25.321884 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:25.322291 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:25.321987 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:26.321925 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:26.321894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:26.322429 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:26.322013 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:27.323017 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:27.322976 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:27.323510 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:27.323103 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:27.323579 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:27.323514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:27.323631 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:27.323607 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:28.321932 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:28.321898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:28.322166 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:28.322009 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:29.321666 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.321641 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:29.322465 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.321649 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:29.322465 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:29.321750 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:29.322465 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:29.321825 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:29.498945 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.498919 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8fqbw" event={"ID":"6d774ec5-c014-4267-92bc-9857e91e8dab","Type":"ContainerStarted","Data":"d466f123ea697871591cf73dc375f1e1961e0ed4d6d9b89892b03b5d6fd9e6c7"} Jul 08 18:18:29.503654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.503553 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"ab430ec5ad102bc6456d335e7d07a05bec2627c4bc95b4d12e06f610d78c916f"} Jul 08 18:18:29.503654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.503613 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"a7c363a2d5bb5e2ec14f5e0575268ddfcb9827a288af7e3ec5b8e983b60dbb19"} Jul 08 18:18:29.503654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.503644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"cacff8f5caf370880a58369f538fdf7fdf6eaaf8d41302ac3a252d2a09702256"} Jul 08 18:18:29.503817 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.503671 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"d4955ea45e447a9e63228e270e5886581ed09f3b8e00b61862eed6e607280f21"} Jul 08 18:18:29.504836 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.504801 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" event={"ID":"cf7d0626-8a75-4e39-adbe-c4e04a9e2e5e","Type":"ContainerStarted","Data":"26bff23a5c4521dbe8a05902cfaa9a1f49b085ac0b7b1d3a554181dabf36a0c8"} Jul 08 18:18:29.513776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.513724 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" event={"ID":"188ef7b029f22b3628e6f2183332d139","Type":"ContainerStarted","Data":"794d33dfaf71c46a5f51efddc7731b9f28afee2cb77a02fb8dc49a796066ff2c"} Jul 08 18:18:29.521178 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.521141 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8fqbw" podStartSLOduration=2.369860113 podStartE2EDuration="22.521129334s" totalImagesPullingTime="20.151269221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:29.520979434 +0000 UTC m=+22.783045186" watchObservedRunningTime="2026-07-08 18:18:29.521129334 +0000 UTC m=+22.783195085" Jul 08 18:18:29.538980 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.538943 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rk8vt" podStartSLOduration=2.283118452 podStartE2EDuration="22.538932262s" totalImagesPullingTime="20.25581381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:29.538434322 +0000 UTC m=+22.800500073" watchObservedRunningTime="2026-07-08 18:18:29.538932262 +0000 UTC m=+22.800998014" Jul 08 18:18:29.552655 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:29.552597 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-106.ec2.internal" podStartSLOduration=21.55258439 podStartE2EDuration="21.55258439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:29.552167136 +0000 UTC m=+22.814232887" watchObservedRunningTime="2026-07-08 18:18:29.55258439 +0000 UTC m=+22.814650141" Jul 08 18:18:30.321922 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.321687 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:30.322458 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:30.321985 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:30.518015 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.517976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-76cqb" event={"ID":"4298d484-fff8-4835-8fdd-d409e875fcaa","Type":"ContainerStarted","Data":"d3dc5e12425ceb8ef3f7c49c79ea214abf98386c1f4491bafb96d6d757a288ac"} Jul 08 18:18:30.519433 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.519408 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wmnmp" event={"ID":"acec3e89-3c81-41eb-a93b-72c59f4eb441","Type":"ContainerStarted","Data":"fb458feba67e7f0bcd0b1be83ae9ca088be4a22380e03dd35e67fb6c0214cc6d"} Jul 08 18:18:30.522659 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.522632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"226729f287d0df9a76abd6043bf71688df016b3351b137aba03d4103599213c2"} Jul 08 18:18:30.522784 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.522701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"50c09ece839a60cd10c6d7d6d9f030a9b14b34ea4d002d3bb7cf05d8d361b613"} Jul 08 18:18:30.524168 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.524147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5fds" event={"ID":"8bfb8749-101a-4ec2-a4ee-19f590ad1350","Type":"ContainerStarted","Data":"7cad6f1f06f4d251961c73cf02de177d8bfcfb63adfa40ee3ea0de68bedd1a64"} Jul 08 18:18:30.525698 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.525670 2562 generic.go:356] "Generic (PLEG): container finished" podID="dee390bc22bbe9a006e589c9e7a83c0e" containerID="1ec03efeca2e6b9fa31308c763c3257517a49e8854da3fe2aaa2dbed41c1a0ff" exitCode=0 Jul 08 18:18:30.525804 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.525746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" event={"ID":"dee390bc22bbe9a006e589c9e7a83c0e","Type":"ContainerDied","Data":"1ec03efeca2e6b9fa31308c763c3257517a49e8854da3fe2aaa2dbed41c1a0ff"} Jul 08 18:18:30.527318 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.527295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-shn8r" event={"ID":"1cc41f43-1ed9-4045-923d-a4b5c47cd31a","Type":"ContainerStarted","Data":"98742b26ce17e35dc76e83c051a7a69a17fa7e3ee6c087e9c75b7f8512de8a6b"} Jul 08 18:18:30.528662 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.528641 2562 generic.go:356] "Generic (PLEG): container finished" podID="beb08a44-9fdd-42a2-9366-a6bcbeb85cb5" containerID="f1ec35ec7e6b4c3c1b3044ee36f2939a6c6d7de9aa79ddc4efc8824cf4252c18" exitCode=0 Jul 08 18:18:30.528786 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.528712 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerDied","Data":"f1ec35ec7e6b4c3c1b3044ee36f2939a6c6d7de9aa79ddc4efc8824cf4252c18"} Jul 08 18:18:30.530622 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.530603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" event={"ID":"7e3da2e7-0256-4012-8b64-129f535b3358","Type":"ContainerStarted","Data":"2f7dd091f3d401b98fc9d9a220833dfe3ac5b865ca3be2b71b26243eee6a26ce"} Jul 08 18:18:30.533370 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.533334 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-76cqb" podStartSLOduration=3.53481668 podStartE2EDuration="23.533323649s" totalImagesPullingTime="19.998506969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:30.533239502 +0000 UTC m=+23.795305254" watchObservedRunningTime="2026-07-08 18:18:30.533323649 +0000 UTC m=+23.795389400" Jul 08 18:18:30.564090 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.564048 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wmnmp" podStartSLOduration=3.533535987 podStartE2EDuration="23.564035087s" totalImagesPullingTime="20.0304991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:30.563452526 +0000 UTC m=+23.825518277" watchObservedRunningTime="2026-07-08 18:18:30.564035087 +0000 UTC m=+23.826100839" Jul 08 18:18:30.578350 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.578253 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v5fds" podStartSLOduration=3.5163555779999998 podStartE2EDuration="23.578240637s" totalImagesPullingTime="20.061885059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:30.578105616 +0000 UTC m=+23.840171362" watchObservedRunningTime="2026-07-08 18:18:30.578240637 +0000 UTC m=+23.840306388" Jul 08 18:18:30.623610 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:30.623535 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-shn8r" podStartSLOduration=3.4815690630000002 podStartE2EDuration="23.623516104s" totalImagesPullingTime="20.141947041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:30.623129872 +0000 UTC m=+23.885195625" watchObservedRunningTime="2026-07-08 18:18:30.623516104 +0000 UTC m=+23.885581857" Jul 08 18:18:31.025993 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.025968 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 18:18:31.192766 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.192569 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T18:18:31.025987338Z","UUID":"c7480e31-28e5-44af-b1e5-1311ed3d99b2","Handler":null,"Name":"","Endpoint":""} Jul 08 18:18:31.194444 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.194419 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 18:18:31.194571 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.194473 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 18:18:31.322280 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.322233 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:31.322712 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:31.322375 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:31.322712 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.322470 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:31.322712 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:31.322590 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:31.534592 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.534547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" event={"ID":"dee390bc22bbe9a006e589c9e7a83c0e","Type":"ContainerStarted","Data":"cf16542753005214632c3eb0b0840ca3053212138040d4c12b47e7aae1e6d901"} Jul 08 18:18:31.536847 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.536817 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" event={"ID":"7e3da2e7-0256-4012-8b64-129f535b3358","Type":"ContainerStarted","Data":"67b5ad51afc3cc48f1990714ee1989e2a07ffba927f98661ef243741da0d38fb"} Jul 08 18:18:31.552878 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:31.552822 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-106.ec2.internal" podStartSLOduration=23.552807589 podStartE2EDuration="23.552807589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:31.552120355 +0000 UTC m=+24.814186110" watchObservedRunningTime="2026-07-08 18:18:31.552807589 +0000 UTC m=+24.814873342" Jul 08 18:18:32.102827 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.102746 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:32.103500 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.103478 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:32.321595 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.321577 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:32.321718 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:32.321685 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:32.542145 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.542108 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" event={"ID":"7e3da2e7-0256-4012-8b64-129f535b3358","Type":"ContainerStarted","Data":"8fb4069aab0b414b488db38fe72ad85102ed0c0d8ca6b2cf5e447762b12f057d"} Jul 08 18:18:32.547859 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.547819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"65c4ed3e93ba0576f49a63a0bda2534dedc1c656ca0ad97fe3e9aa8b4d010075"} Jul 08 18:18:32.548182 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.548149 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:32.548551 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.548533 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wmnmp" Jul 08 18:18:32.560881 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:32.560822 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b4wpb" podStartSLOduration=3.118577844 podStartE2EDuration="25.560802385s" totalImagesPullingTime="22.442224541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:32.560709161 +0000 UTC m=+25.822774915" watchObservedRunningTime="2026-07-08 18:18:32.560802385 +0000 UTC m=+25.822868137" Jul 08 18:18:33.322393 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:33.322324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:33.322571 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:33.322324 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:33.322571 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:33.322455 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:33.322571 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:33.322542 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:34.322280 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:34.322237 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:34.322728 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:34.322390 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:35.322070 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.321880 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:35.322250 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.321880 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:35.322250 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:35.322164 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:35.322250 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:35.322213 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:35.558407 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.558367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" event={"ID":"f1ed1352-d8a0-4c55-a504-8cf3cc12b2ce","Type":"ContainerStarted","Data":"ed9b95ce0ed11624a07f691bccabdd2f551c7ce18266b7d9829af4c85a8074cc"} Jul 08 18:18:35.558680 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.558618 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:35.558783 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.558733 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:35.560496 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.560460 2562 generic.go:356] "Generic (PLEG): container finished" podID="beb08a44-9fdd-42a2-9366-a6bcbeb85cb5" containerID="18b423eb2f6f77d859b3b04e4b52964582631e1927a816c5072c91379ee00b09" exitCode=0 Jul 08 18:18:35.560496 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.560490 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerDied","Data":"18b423eb2f6f77d859b3b04e4b52964582631e1927a816c5072c91379ee00b09"} Jul 08 18:18:35.575200 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.575179 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:35.587605 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:35.587531 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" podStartSLOduration=7.9723863470000005 podStartE2EDuration="28.587517578s" totalImagesPullingTime="20.615131231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:35.587154361 +0000 UTC m=+28.849220114" watchObservedRunningTime="2026-07-08 18:18:35.587517578 +0000 UTC m=+28.849583329" Jul 08 18:18:36.321644 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.321448 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:36.321966 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:36.321743 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:36.563308 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.563279 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:36.578077 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.578014 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:18:36.956523 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.956449 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cgqjt"] Jul 08 18:18:36.956681 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.956577 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:36.956765 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:36.956705 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:36.958822 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.958791 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-597wz"] Jul 08 18:18:36.958990 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.958892 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:36.958990 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:36.958963 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:36.968553 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.968534 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j2f2w"] Jul 08 18:18:36.968640 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:36.968609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:36.968718 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:36.968686 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:37.567404 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:37.567366 2562 generic.go:356] "Generic (PLEG): container finished" podID="beb08a44-9fdd-42a2-9366-a6bcbeb85cb5" containerID="01582da23cfbe8319aeab890f8141e3ded791679419d625902b9579f442e17c6" exitCode=0 Jul 08 18:18:37.567856 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:37.567459 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerDied","Data":"01582da23cfbe8319aeab890f8141e3ded791679419d625902b9579f442e17c6"} Jul 08 18:18:38.321654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:38.321630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:38.321759 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:38.321728 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:38.321759 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:38.321753 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:38.321893 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:38.321810 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:38.573290 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:38.573242 2562 generic.go:356] "Generic (PLEG): container finished" podID="beb08a44-9fdd-42a2-9366-a6bcbeb85cb5" containerID="ec8efc4187fd4ed839f796fb9dc10ddac42283bbb48a082c6d2fcd1734206adf" exitCode=0 Jul 08 18:18:38.573874 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:38.573313 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerDied","Data":"ec8efc4187fd4ed839f796fb9dc10ddac42283bbb48a082c6d2fcd1734206adf"} Jul 08 18:18:39.321706 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:39.321472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:39.321863 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:39.321773 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:39.980381 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:39.980341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:39.980871 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:39.980423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:39.980871 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:39.980502 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:39.980871 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:39.980551 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:39.980871 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:39.980576 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret podName:b59bbc9f-1106-47b1-ae05-4ebc641b1249 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:11.980556357 +0000 UTC m=+65.242622111 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret") pod "global-pull-secret-syncer-j2f2w" (UID: "b59bbc9f-1106-47b1-ae05-4ebc641b1249") : object "kube-system"/"original-pull-secret" not registered Jul 08 18:18:39.980871 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:39.980605 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:11.980586859 +0000 UTC m=+65.242652589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 18:18:40.080860 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:40.080818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:40.081035 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:40.080990 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 18:18:40.081035 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:40.081015 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 18:18:40.081035 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:40.081031 2562 projected.go:196] Error preparing data for projected volume kube-api-access-cp2v5 for pod openshift-network-diagnostics/network-check-target-597wz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:40.081191 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:40.081094 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5 podName:ab4d0b34-0445-427e-8ce3-5b1c5595f7de nodeName:}" failed. No retries permitted until 2026-07-08 18:19:12.081077554 +0000 UTC m=+65.343143304 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cp2v5" (UniqueName: "kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5") pod "network-check-target-597wz" (UID: "ab4d0b34-0445-427e-8ce3-5b1c5595f7de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 18:18:40.322258 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:40.322185 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:40.322258 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:40.322216 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:40.322450 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:40.322318 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-597wz" podUID="ab4d0b34-0445-427e-8ce3-5b1c5595f7de" Jul 08 18:18:40.322450 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:40.322434 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:18:41.321911 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.321876 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:41.322393 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:41.322003 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j2f2w" podUID="b59bbc9f-1106-47b1-ae05-4ebc641b1249" Jul 08 18:18:41.505443 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.505412 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-106.ec2.internal" event="NodeReady" Jul 08 18:18:41.505659 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.505562 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 18:18:41.549991 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.549961 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5gl29"] Jul 08 18:18:41.552709 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.552675 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vnlvt"] Jul 08 18:18:41.552843 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.552789 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.554488 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.554468 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.555294 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.555276 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4sdr\"" Jul 08 18:18:41.555638 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.555620 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 18:18:41.555721 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.555641 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 18:18:41.558144 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.558125 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 18:18:41.558247 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.558145 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 18:18:41.558247 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.558146 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 18:18:41.558620 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.558597 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r6bqm\"" Jul 08 18:18:41.563656 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.563638 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5gl29"] Jul 08 18:18:41.567188 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.567162 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vnlvt"] Jul 08 18:18:41.691133 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.691030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/321d20ce-3b8f-4775-9cb6-3f2793a9a235-tmp-dir\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.691133 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.691084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/321d20ce-3b8f-4775-9cb6-3f2793a9a235-config-volume\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.691369 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.691189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43a6c05e-9764-4048-86c8-4f8aaa99d812-cert\") pod \"ingress-canary-vnlvt\" (UID: \"43a6c05e-9764-4048-86c8-4f8aaa99d812\") " pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.691369 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.691230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqsdc\" (UniqueName: \"kubernetes.io/projected/43a6c05e-9764-4048-86c8-4f8aaa99d812-kube-api-access-mqsdc\") pod \"ingress-canary-vnlvt\" (UID: \"43a6c05e-9764-4048-86c8-4f8aaa99d812\") " pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.691369 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.691345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmzlh\" (UniqueName: \"kubernetes.io/projected/321d20ce-3b8f-4775-9cb6-3f2793a9a235-kube-api-access-fmzlh\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.691505 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.691386 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.792383 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.792341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmzlh\" (UniqueName: \"kubernetes.io/projected/321d20ce-3b8f-4775-9cb6-3f2793a9a235-kube-api-access-fmzlh\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.792568 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.792411 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.792568 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.792437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/321d20ce-3b8f-4775-9cb6-3f2793a9a235-tmp-dir\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.792568 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.792456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/321d20ce-3b8f-4775-9cb6-3f2793a9a235-config-volume\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.792568 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.792487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43a6c05e-9764-4048-86c8-4f8aaa99d812-cert\") pod \"ingress-canary-vnlvt\" (UID: \"43a6c05e-9764-4048-86c8-4f8aaa99d812\") " pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.792792 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:41.792631 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:41.792792 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:41.792728 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:42.292705 +0000 UTC m=+35.554770746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:18:41.792892 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.792790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqsdc\" (UniqueName: \"kubernetes.io/projected/43a6c05e-9764-4048-86c8-4f8aaa99d812-kube-api-access-mqsdc\") pod \"ingress-canary-vnlvt\" (UID: \"43a6c05e-9764-4048-86c8-4f8aaa99d812\") " pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.793152 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.793131 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/321d20ce-3b8f-4775-9cb6-3f2793a9a235-tmp-dir\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.793388 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.793359 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/321d20ce-3b8f-4775-9cb6-3f2793a9a235-config-volume\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.797527 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.797502 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43a6c05e-9764-4048-86c8-4f8aaa99d812-cert\") pod \"ingress-canary-vnlvt\" (UID: \"43a6c05e-9764-4048-86c8-4f8aaa99d812\") " pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.803423 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.803386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqsdc\" (UniqueName: \"kubernetes.io/projected/43a6c05e-9764-4048-86c8-4f8aaa99d812-kube-api-access-mqsdc\") pod \"ingress-canary-vnlvt\" (UID: \"43a6c05e-9764-4048-86c8-4f8aaa99d812\") " pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:41.804603 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.804582 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmzlh\" (UniqueName: \"kubernetes.io/projected/321d20ce-3b8f-4775-9cb6-3f2793a9a235-kube-api-access-fmzlh\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:41.871827 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:41.871783 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vnlvt" Jul 08 18:18:42.297562 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.297527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:42.297745 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:42.297656 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:42.297745 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:42.297713 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:43.297698262 +0000 UTC m=+36.559763996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:18:42.321570 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.321537 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:18:42.321570 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.321555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:18:42.325533 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.325503 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:18:42.326102 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.325540 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9qsc4\"" Jul 08 18:18:42.326102 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.325628 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:18:42.326102 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.325543 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:18:42.326102 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.325503 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcpkw\"" Jul 08 18:18:42.595911 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:42.595830 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vnlvt"] Jul 08 18:18:42.678756 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:42.678723 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a6c05e_9764_4048_86c8_4f8aaa99d812.slice/crio-08d1f5b8b89e922b20848a56986a21b81f323526cff5fbe80983e20427359b0f WatchSource:0}: Error finding container 08d1f5b8b89e922b20848a56986a21b81f323526cff5fbe80983e20427359b0f: Status 404 returned error can't find the container with id 08d1f5b8b89e922b20848a56986a21b81f323526cff5fbe80983e20427359b0f Jul 08 18:18:43.306058 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:43.305844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:43.306234 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:43.306019 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:43.306234 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:43.306138 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:45.30612076 +0000 UTC m=+38.568186492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:18:43.321537 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:43.321506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:18:43.324331 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:43.324312 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:18:43.589334 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:43.589233 2562 generic.go:356] "Generic (PLEG): container finished" podID="beb08a44-9fdd-42a2-9366-a6bcbeb85cb5" containerID="5032762fdc39611caff051e38ca6a5a0b26af2e8defa72301797e2d7a0475874" exitCode=0 Jul 08 18:18:43.589334 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:43.589299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerDied","Data":"5032762fdc39611caff051e38ca6a5a0b26af2e8defa72301797e2d7a0475874"} Jul 08 18:18:43.590743 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:43.590704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vnlvt" event={"ID":"43a6c05e-9764-4048-86c8-4f8aaa99d812","Type":"ContainerStarted","Data":"08d1f5b8b89e922b20848a56986a21b81f323526cff5fbe80983e20427359b0f"} Jul 08 18:18:44.596700 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:44.596654 2562 generic.go:356] "Generic (PLEG): container finished" podID="beb08a44-9fdd-42a2-9366-a6bcbeb85cb5" containerID="09a1c18db8bb4b6e2bbfbb5794e78aa60600ed13a3e772d8365b5b5bd5ad375e" exitCode=0 Jul 08 18:18:44.597105 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:44.596720 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerDied","Data":"09a1c18db8bb4b6e2bbfbb5794e78aa60600ed13a3e772d8365b5b5bd5ad375e"} Jul 08 18:18:45.319811 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:45.319774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:45.319972 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:45.319924 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:45.320012 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:45.320002 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:49.319976462 +0000 UTC m=+42.582042192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:18:45.602820 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:45.602786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mjcct" event={"ID":"beb08a44-9fdd-42a2-9366-a6bcbeb85cb5","Type":"ContainerStarted","Data":"2d560f121562e2d8d9cd7fb20483cc6628a54a123495dbdc5f4c09250eed2260"} Jul 08 18:18:45.604180 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:45.604159 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vnlvt" event={"ID":"43a6c05e-9764-4048-86c8-4f8aaa99d812","Type":"ContainerStarted","Data":"e4db926b99904a4d452016e7d5a13b4fb582f331fa28498e182b79cec847d84c"} Jul 08 18:18:45.630291 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:45.630223 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mjcct" podStartSLOduration=8.178102199 podStartE2EDuration="38.630204622s" totalImagesPullingTime="30.452102423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:45.62848067 +0000 UTC m=+38.890546419" watchObservedRunningTime="2026-07-08 18:18:45.630204622 +0000 UTC m=+38.892270374" Jul 08 18:18:45.647990 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:45.647949 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vnlvt" podStartSLOduration=2.229315018 podStartE2EDuration="4.647933007s" totalImagesPullingTime="2.418617989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:45.64659303 +0000 UTC m=+38.908658782" watchObservedRunningTime="2026-07-08 18:18:45.647933007 +0000 UTC m=+38.909998758" Jul 08 18:18:49.345494 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.345312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:49.345867 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:49.345452 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:49.345867 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:49.345615 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:18:57.345598133 +0000 UTC m=+50.607663866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:18:49.510849 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.510819 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk"] Jul 08 18:18:49.534996 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.534970 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft"] Jul 08 18:18:49.535156 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.535093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.537822 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.537799 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 18:18:49.537939 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.537891 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 18:18:49.538280 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.538250 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 18:18:49.538365 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.538296 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 18:18:49.548871 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.548853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk"] Jul 08 18:18:49.548944 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.548885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft"] Jul 08 18:18:49.548988 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.548954 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.551951 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.551932 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 18:18:49.552174 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.552157 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 18:18:49.552245 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.552186 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 18:18:49.552431 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.552294 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 18:18:49.648459 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648386 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.648459 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648429 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plwmp\" (UniqueName: \"kubernetes.io/projected/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-kube-api-access-plwmp\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.648627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648468 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f739ccf3-7738-4f06-a194-5fe46b609a74-klusterlet-config\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.648627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648484 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.648627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648524 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zhc6\" (UniqueName: \"kubernetes.io/projected/f739ccf3-7738-4f06-a194-5fe46b609a74-kube-api-access-9zhc6\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.648627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f739ccf3-7738-4f06-a194-5fe46b609a74-tmp\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.648627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648606 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-hub\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.648814 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648623 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.648814 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.648673 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-ca\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.749430 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plwmp\" (UniqueName: \"kubernetes.io/projected/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-kube-api-access-plwmp\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.749598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f739ccf3-7738-4f06-a194-5fe46b609a74-klusterlet-config\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.749598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.749598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zhc6\" (UniqueName: \"kubernetes.io/projected/f739ccf3-7738-4f06-a194-5fe46b609a74-kube-api-access-9zhc6\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.749598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749559 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f739ccf3-7738-4f06-a194-5fe46b609a74-tmp\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.749598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-hub\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.749772 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.749682 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.750355 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.750028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-ca\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.750355 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.750098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.750355 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.750290 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f739ccf3-7738-4f06-a194-5fe46b609a74-tmp\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.752677 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.750877 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.755341 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.755253 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.755341 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.755303 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-ca\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.755341 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.755312 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-hub\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.755545 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.755417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f739ccf3-7738-4f06-a194-5fe46b609a74-klusterlet-config\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.757022 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.757002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.758758 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.758731 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plwmp\" (UniqueName: \"kubernetes.io/projected/e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872-kube-api-access-plwmp\") pod \"cluster-proxy-proxy-agent-558fc8966b-g87ft\" (UID: \"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:49.758850 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.758831 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zhc6\" (UniqueName: \"kubernetes.io/projected/f739ccf3-7738-4f06-a194-5fe46b609a74-kube-api-access-9zhc6\") pod \"klusterlet-addon-workmgr-795d4db5fb-5h6lk\" (UID: \"f739ccf3-7738-4f06-a194-5fe46b609a74\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.845099 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.845069 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:49.865402 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:49.865374 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:18:50.008998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:50.008971 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk"] Jul 08 18:18:50.010922 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:50.010897 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf739ccf3_7738_4f06_a194_5fe46b609a74.slice/crio-5e9be2c7d48b5ba34743c5a3793cfc62c014a5ca780d3dcf3a49ed0bc93a9177 WatchSource:0}: Error finding container 5e9be2c7d48b5ba34743c5a3793cfc62c014a5ca780d3dcf3a49ed0bc93a9177: Status 404 returned error can't find the container with id 5e9be2c7d48b5ba34743c5a3793cfc62c014a5ca780d3dcf3a49ed0bc93a9177 Jul 08 18:18:50.027408 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:50.027369 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft"] Jul 08 18:18:50.030072 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:18:50.030050 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode68f0ed0_48aa_4f8b_b7e9_4f9b4751f872.slice/crio-2800f1be11f6e7ef333d62d798aa7d53252d018c00b44a6a10b31202f955282b WatchSource:0}: Error finding container 2800f1be11f6e7ef333d62d798aa7d53252d018c00b44a6a10b31202f955282b: Status 404 returned error can't find the container with id 2800f1be11f6e7ef333d62d798aa7d53252d018c00b44a6a10b31202f955282b Jul 08 18:18:50.617239 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:50.617199 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" event={"ID":"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872","Type":"ContainerStarted","Data":"2800f1be11f6e7ef333d62d798aa7d53252d018c00b44a6a10b31202f955282b"} Jul 08 18:18:50.618377 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:50.618351 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" event={"ID":"f739ccf3-7738-4f06-a194-5fe46b609a74","Type":"ContainerStarted","Data":"5e9be2c7d48b5ba34743c5a3793cfc62c014a5ca780d3dcf3a49ed0bc93a9177"} Jul 08 18:18:56.638049 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.638009 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" event={"ID":"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872","Type":"ContainerStarted","Data":"f9d7d64b904e1f52b1bed740d34df62add725d47dce3e8d5806716ad0c8b5fc1"} Jul 08 18:18:56.638506 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.638059 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" event={"ID":"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872","Type":"ContainerStarted","Data":"2e7424836c9a9127203f30f4367a1a1ea6ecce336e76659c263107eb9117834f"} Jul 08 18:18:56.638506 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.638076 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" event={"ID":"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872","Type":"ContainerStarted","Data":"b1f51d4565d208b0ade25194db5ae6b81014287b748d8df53bbf87694c22a582"} Jul 08 18:18:56.639572 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.639547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" event={"ID":"f739ccf3-7738-4f06-a194-5fe46b609a74","Type":"ContainerStarted","Data":"6d211665ee05d819d4cac41633a607940cdc452d76e6b36488772981dc548f34"} Jul 08 18:18:56.639683 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.639629 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:56.641400 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.641374 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:18:56.656861 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.656821 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" podStartSLOduration=2.01770957 podStartE2EDuration="7.656801547s" totalImagesPullingTime="5.639091977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:56.656439459 +0000 UTC m=+49.918505214" watchObservedRunningTime="2026-07-08 18:18:56.656801547 +0000 UTC m=+49.918867299" Jul 08 18:18:56.671795 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:56.671758 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" podStartSLOduration=2.028528776 podStartE2EDuration="7.671747491s" totalImagesPullingTime="5.643218715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:18:56.671333531 +0000 UTC m=+49.933399303" watchObservedRunningTime="2026-07-08 18:18:56.671747491 +0000 UTC m=+49.933813243" Jul 08 18:18:57.412361 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:18:57.412328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:18:57.412546 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:57.412448 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:18:57.412546 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:18:57.412498 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:13.412483735 +0000 UTC m=+66.674549465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:19:08.584252 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:08.584220 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jfzsd" Jul 08 18:19:12.013798 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.013763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:19:12.014287 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.013845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:19:12.016289 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.016251 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 18:19:12.016343 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.016250 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 18:19:12.024314 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:19:12.024281 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:19:12.024441 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:19:12.024349 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:20:16.024333543 +0000 UTC m=+129.286399276 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : secret "metrics-daemon-secret" not found Jul 08 18:19:12.026511 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.026489 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b59bbc9f-1106-47b1-ae05-4ebc641b1249-original-pull-secret\") pod \"global-pull-secret-syncer-j2f2w\" (UID: \"b59bbc9f-1106-47b1-ae05-4ebc641b1249\") " pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:19:12.114418 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.114390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:19:12.116984 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.116962 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 18:19:12.126948 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.126926 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 18:19:12.132813 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.132798 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j2f2w" Jul 08 18:19:12.138218 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.138202 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp2v5\" (UniqueName: \"kubernetes.io/projected/ab4d0b34-0445-427e-8ce3-5b1c5595f7de-kube-api-access-cp2v5\") pod \"network-check-target-597wz\" (UID: \"ab4d0b34-0445-427e-8ce3-5b1c5595f7de\") " pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:19:12.277913 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.277836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j2f2w"] Jul 08 18:19:12.281027 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:19:12.281000 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59bbc9f_1106_47b1_ae05_4ebc641b1249.slice/crio-55db062703842ed35e9f749ef6e70aab3e95780641e4769f077d3edf01eb9cd7 WatchSource:0}: Error finding container 55db062703842ed35e9f749ef6e70aab3e95780641e4769f077d3edf01eb9cd7: Status 404 returned error can't find the container with id 55db062703842ed35e9f749ef6e70aab3e95780641e4769f077d3edf01eb9cd7 Jul 08 18:19:12.342307 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.342284 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kcpkw\"" Jul 08 18:19:12.350391 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.350375 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:19:12.470619 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.470592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-597wz"] Jul 08 18:19:12.473613 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:19:12.473593 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4d0b34_0445_427e_8ce3_5b1c5595f7de.slice/crio-81c90334dc7d29923f6f28990aa5c7ca6a91e1f4d3775eda364c590e2dc72976 WatchSource:0}: Error finding container 81c90334dc7d29923f6f28990aa5c7ca6a91e1f4d3775eda364c590e2dc72976: Status 404 returned error can't find the container with id 81c90334dc7d29923f6f28990aa5c7ca6a91e1f4d3775eda364c590e2dc72976 Jul 08 18:19:12.680259 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.680191 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-597wz" event={"ID":"ab4d0b34-0445-427e-8ce3-5b1c5595f7de","Type":"ContainerStarted","Data":"81c90334dc7d29923f6f28990aa5c7ca6a91e1f4d3775eda364c590e2dc72976"} Jul 08 18:19:12.681166 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:12.681137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j2f2w" event={"ID":"b59bbc9f-1106-47b1-ae05-4ebc641b1249","Type":"ContainerStarted","Data":"55db062703842ed35e9f749ef6e70aab3e95780641e4769f077d3edf01eb9cd7"} Jul 08 18:19:13.423785 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:13.423747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:19:13.424255 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:19:13.423919 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:19:13.424255 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:19:13.423983 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:19:45.423963439 +0000 UTC m=+98.686029191 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:19:17.699616 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:17.699575 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-597wz" event={"ID":"ab4d0b34-0445-427e-8ce3-5b1c5595f7de","Type":"ContainerStarted","Data":"2d6a9b519e40c1d9c9d77c28b962bba92b1bdd0f360406e013694066dc03c18b"} Jul 08 18:19:17.700088 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:17.699662 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:19:17.701108 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:17.701077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j2f2w" event={"ID":"b59bbc9f-1106-47b1-ae05-4ebc641b1249","Type":"ContainerStarted","Data":"93528814c03679879ac3a372caa64e58d7b9248e1cb5f6e851007e7d5b1cf461"} Jul 08 18:19:17.718970 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:17.718909 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-597wz" podStartSLOduration=66.296961423 podStartE2EDuration="1m10.718891185s" totalImagesPullingTime="4.421929762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:19:17.717033468 +0000 UTC m=+70.979099247" watchObservedRunningTime="2026-07-08 18:19:17.718891185 +0000 UTC m=+70.980956939" Jul 08 18:19:17.732345 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:17.732301 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j2f2w" podStartSLOduration=66.114839375 podStartE2EDuration="1m10.73228693s" totalImagesPullingTime="4.617447555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:19:17.731955665 +0000 UTC m=+70.994021414" watchObservedRunningTime="2026-07-08 18:19:17.73228693 +0000 UTC m=+70.994352684" Jul 08 18:19:45.436876 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:45.436745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:19:45.436876 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:19:45.436860 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:19:45.437456 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:19:45.436930 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:20:49.436912142 +0000 UTC m=+162.698977888 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:19:48.706014 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:19:48.705984 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-597wz" Jul 08 18:20:16.045491 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:16.045445 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:20:16.045967 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:20:16.045575 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:20:16.045967 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:20:16.045637 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:22:18.045622507 +0000 UTC m=+251.307688236 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : secret "metrics-daemon-secret" not found Jul 08 18:20:44.565117 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:20:44.565062 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5gl29" podUID="321d20ce-3b8f-4775-9cb6-3f2793a9a235" Jul 08 18:20:44.949440 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:44.949360 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5gl29" Jul 08 18:20:45.333428 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:20:45.333394 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:20:49.481982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:49.481929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:20:49.482393 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:20:49.482071 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 18:20:49.482393 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:20:49.482133 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls podName:321d20ce-3b8f-4775-9cb6-3f2793a9a235 nodeName:}" failed. No retries permitted until 2026-07-08 18:22:51.482117629 +0000 UTC m=+284.744183358 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls") pod "dns-default-5gl29" (UID: "321d20ce-3b8f-4775-9cb6-3f2793a9a235") : secret "dns-default-metrics-tls" not found Jul 08 18:20:55.981033 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:55.980999 2562 generic.go:356] "Generic (PLEG): container finished" podID="f739ccf3-7738-4f06-a194-5fe46b609a74" containerID="6d211665ee05d819d4cac41633a607940cdc452d76e6b36488772981dc548f34" exitCode=1 Jul 08 18:20:55.981466 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:55.981041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" event={"ID":"f739ccf3-7738-4f06-a194-5fe46b609a74","Type":"ContainerDied","Data":"6d211665ee05d819d4cac41633a607940cdc452d76e6b36488772981dc548f34"} Jul 08 18:20:55.981466 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:55.981403 2562 scope.go:117] "RemoveContainer" containerID="6d211665ee05d819d4cac41633a607940cdc452d76e6b36488772981dc548f34" Jul 08 18:20:56.640114 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:56.640075 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:20:56.985582 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:56.985550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" event={"ID":"f739ccf3-7738-4f06-a194-5fe46b609a74","Type":"ContainerStarted","Data":"d64a8ac642854acc3c9a0929b3493be8d04e8cc80a06fadfea945bc38bcecba3"} Jul 08 18:20:56.985991 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:56.985805 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:20:56.986458 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:56.986442 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-795d4db5fb-5h6lk" Jul 08 18:20:57.323248 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:20:57.323168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:22:17.013172 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.013137 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6btwg"] Jul 08 18:22:17.015243 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.015225 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.017884 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.017863 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 18:22:17.018819 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.018799 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 18:22:17.018904 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.018868 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b7xvt\"" Jul 08 18:22:17.018968 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.018935 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 18:22:17.019302 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.019289 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 18:22:17.028214 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.028191 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6btwg"] Jul 08 18:22:17.088193 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.088155 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.088193 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.088207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.088446 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.088228 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnnkc\" (UniqueName: \"kubernetes.io/projected/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-api-access-tnnkc\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.088446 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.088255 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd8fca41-b253-46eb-95cf-554d8657d33d-data-volume\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.088446 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.088387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd8fca41-b253-46eb-95cf-554d8657d33d-crio-socket\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.188998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.188956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.188998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.189006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.189191 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.189026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnnkc\" (UniqueName: \"kubernetes.io/projected/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-api-access-tnnkc\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.189191 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.189090 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:17.189191 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.189112 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.189191 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.189147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd8fca41-b253-46eb-95cf-554d8657d33d-data-volume\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.189191 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.189166 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.689148383 +0000 UTC m=+250.951214125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : configmap "kube-rbac-proxy" not found Jul 08 18:22:17.189390 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.189224 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:17.689207549 +0000 UTC m=+250.951273278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.189390 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.189318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd8fca41-b253-46eb-95cf-554d8657d33d-crio-socket\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.189457 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.189397 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd8fca41-b253-46eb-95cf-554d8657d33d-crio-socket\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.189507 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.189488 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd8fca41-b253-46eb-95cf-554d8657d33d-data-volume\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.197677 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.197659 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnnkc\" (UniqueName: \"kubernetes.io/projected/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-api-access-tnnkc\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.693137 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.693106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.693353 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:17.693161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:17.693353 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.693281 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:17.693353 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.693328 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:17.693523 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.693376 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:18.693351971 +0000 UTC m=+251.955417701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : configmap "kube-rbac-proxy" not found Jul 08 18:22:17.693523 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:17.693398 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:18.693387153 +0000 UTC m=+251.955452889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:18.096586 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:18.096545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:22:18.097047 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:18.096683 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 18:22:18.097047 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:18.096769 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs podName:31aa72aa-4b52-4112-a90a-f8b9f4784008 nodeName:}" failed. No retries permitted until 2026-07-08 18:24:20.09674961 +0000 UTC m=+373.358815343 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs") pod "network-metrics-daemon-cgqjt" (UID: "31aa72aa-4b52-4112-a90a-f8b9f4784008") : secret "metrics-daemon-secret" not found Jul 08 18:22:18.701990 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:18.701945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:18.701990 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:18.702001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:18.702258 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:18.702096 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:18.702258 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:18.702185 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:20.702164289 +0000 UTC m=+253.964230032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : configmap "kube-rbac-proxy" not found Jul 08 18:22:18.702258 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:18.702105 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:18.702258 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:18.702229 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:20.7022183 +0000 UTC m=+253.964284029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:18.882000 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:18.881973 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-76cqb_4298d484-fff8-4835-8fdd-d409e875fcaa/dns-node-resolver/0.log" Jul 08 18:22:20.279535 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:20.279507 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5fds_8bfb8749-101a-4ec2-a4ee-19f590ad1350/node-ca/0.log" Jul 08 18:22:20.716912 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:20.716823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:20.716912 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:20.716872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:20.717096 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:20.716956 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:20.717096 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:20.717002 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:20.717096 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:20.717025 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:24.717010663 +0000 UTC m=+257.979076394 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : configmap "kube-rbac-proxy" not found Jul 08 18:22:20.717096 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:20.717042 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:24.717032087 +0000 UTC m=+257.979097816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:21.080576 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:21.080548 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vnlvt_43a6c05e-9764-4048-86c8-4f8aaa99d812/serve-healthcheck-canary/0.log" Jul 08 18:22:24.746841 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:24.746794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:24.747427 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:24.746875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:24.747427 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:24.746939 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 18:22:24.747427 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:24.746989 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:24.747427 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:24.747003 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:32.7469886 +0000 UTC m=+266.009054329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : configmap "kube-rbac-proxy" not found Jul 08 18:22:24.747427 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:24.747108 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:32.747089504 +0000 UTC m=+266.009155233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:32.807983 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:32.807890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:32.807983 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:32.807971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:32.808416 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:32.808055 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 18:22:32.808416 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:32.808131 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls podName:cd8fca41-b253-46eb-95cf-554d8657d33d nodeName:}" failed. No retries permitted until 2026-07-08 18:22:48.808112502 +0000 UTC m=+282.070178232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6btwg" (UID: "cd8fca41-b253-46eb-95cf-554d8657d33d") : secret "insights-runtime-extractor-tls" not found Jul 08 18:22:32.808488 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:32.808473 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd8fca41-b253-46eb-95cf-554d8657d33d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:47.950458 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:22:47.950400 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5gl29" podUID="321d20ce-3b8f-4775-9cb6-3f2793a9a235" Jul 08 18:22:48.290612 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:48.290584 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5gl29" Jul 08 18:22:48.815808 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:48.815770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:48.818148 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:48.818128 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd8fca41-b253-46eb-95cf-554d8657d33d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6btwg\" (UID: \"cd8fca41-b253-46eb-95cf-554d8657d33d\") " pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:48.824110 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:48.824089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6btwg" Jul 08 18:22:48.948017 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:48.947989 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6btwg"] Jul 08 18:22:48.951551 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:22:48.951525 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8fca41_b253_46eb_95cf_554d8657d33d.slice/crio-2c3b2de58b197abbe5e289ec2bdfff03b65d0420cff13dc79a9f71a9faf4dfb2 WatchSource:0}: Error finding container 2c3b2de58b197abbe5e289ec2bdfff03b65d0420cff13dc79a9f71a9faf4dfb2: Status 404 returned error can't find the container with id 2c3b2de58b197abbe5e289ec2bdfff03b65d0420cff13dc79a9f71a9faf4dfb2 Jul 08 18:22:49.294332 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:49.294293 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6btwg" event={"ID":"cd8fca41-b253-46eb-95cf-554d8657d33d","Type":"ContainerStarted","Data":"2c3b2de58b197abbe5e289ec2bdfff03b65d0420cff13dc79a9f71a9faf4dfb2"} Jul 08 18:22:51.542251 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:51.542208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:22:51.545067 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:51.545037 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/321d20ce-3b8f-4775-9cb6-3f2793a9a235-metrics-tls\") pod \"dns-default-5gl29\" (UID: \"321d20ce-3b8f-4775-9cb6-3f2793a9a235\") " pod="openshift-dns/dns-default-5gl29" Jul 08 18:22:51.593689 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:51.593660 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d4sdr\"" Jul 08 18:22:51.601906 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:51.601873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5gl29" Jul 08 18:22:52.843576 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:52.843542 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5gl29"] Jul 08 18:22:52.846350 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:22:52.846316 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321d20ce_3b8f_4775_9cb6_3f2793a9a235.slice/crio-009d51e0bd049acfee68e716175cbfe0363d464c6f1c6f806f8378ff01ab2183 WatchSource:0}: Error finding container 009d51e0bd049acfee68e716175cbfe0363d464c6f1c6f806f8378ff01ab2183: Status 404 returned error can't find the container with id 009d51e0bd049acfee68e716175cbfe0363d464c6f1c6f806f8378ff01ab2183 Jul 08 18:22:53.310129 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:53.310087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6btwg" event={"ID":"cd8fca41-b253-46eb-95cf-554d8657d33d","Type":"ContainerStarted","Data":"ec3af826bfabc5eb23e71d51c461a9a5f50d913724cc3b4a6acb47511688f0d4"} Jul 08 18:22:53.311408 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:53.311381 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5gl29" event={"ID":"321d20ce-3b8f-4775-9cb6-3f2793a9a235","Type":"ContainerStarted","Data":"009d51e0bd049acfee68e716175cbfe0363d464c6f1c6f806f8378ff01ab2183"} Jul 08 18:22:54.316070 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:54.316039 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6btwg" event={"ID":"cd8fca41-b253-46eb-95cf-554d8657d33d","Type":"ContainerStarted","Data":"1b57d7dfd141d3de1bdd8b7bfc2802b7a7fbcd0c928f7bef2148a14b73a55adb"} Jul 08 18:22:55.325250 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:55.325214 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5gl29" event={"ID":"321d20ce-3b8f-4775-9cb6-3f2793a9a235","Type":"ContainerStarted","Data":"4194ba15daf57b78335c8e6523f092e58b3b41b35b171e03db7e3dec171f212b"} Jul 08 18:22:55.325649 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:55.325296 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5gl29" Jul 08 18:22:55.325649 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:55.325319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5gl29" event={"ID":"321d20ce-3b8f-4775-9cb6-3f2793a9a235","Type":"ContainerStarted","Data":"5ffb1fe63787984521c24c1be64151c822c338c68dce6d71e46bec05da819b37"} Jul 08 18:22:55.349423 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:55.349363 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5gl29" podStartSLOduration=252.937392318 podStartE2EDuration="4m14.349345784s" totalImagesPullingTime="1.411953466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:55.348441685 +0000 UTC m=+288.610507468" watchObservedRunningTime="2026-07-08 18:22:55.349345784 +0000 UTC m=+288.611411534" Jul 08 18:22:57.330569 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.330534 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6btwg" event={"ID":"cd8fca41-b253-46eb-95cf-554d8657d33d","Type":"ContainerStarted","Data":"85568ce95cce2320797a53db05035974625e22004ecfa3f4525c472e2bd82312"} Jul 08 18:22:57.350776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.350734 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6btwg" podStartSLOduration=34.118453087 podStartE2EDuration="41.350718484s" totalImagesPullingTime="7.232265397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:22:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:22:57.350337529 +0000 UTC m=+290.612403273" watchObservedRunningTime="2026-07-08 18:22:57.350718484 +0000 UTC m=+290.612784213" Jul 08 18:22:57.989305 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.989255 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ddq2n"] Jul 08 18:22:57.991558 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.991542 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:57.994803 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.994777 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 18:22:57.994951 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.994929 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hbm8s\"" Jul 08 18:22:57.995441 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.995416 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 18:22:57.995520 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.995466 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 18:22:57.995679 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.995664 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 18:22:57.995751 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.995735 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 18:22:57.995809 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:57.995795 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 18:22:58.094136 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094101 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-textfile\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094136 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghqfk\" (UniqueName: \"kubernetes.io/projected/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-kube-api-access-ghqfk\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094399 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-sys\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094399 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-wtmp\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094399 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094314 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094399 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-metrics-client-ca\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094399 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094583 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-root\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.094583 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.094437 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-tls\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195051 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-wtmp\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195051 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195051 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-metrics-client-ca\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-root\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-tls\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-textfile\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghqfk\" (UniqueName: \"kubernetes.io/projected/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-kube-api-access-ghqfk\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195213 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-wtmp\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-sys\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195240 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-root\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195319 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-sys\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195738 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-textfile\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195812 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195791 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-metrics-client-ca\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.195887 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.195872 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.197598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.197574 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.197698 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.197639 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-node-exporter-tls\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.206057 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.206033 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghqfk\" (UniqueName: \"kubernetes.io/projected/05245de8-9d7d-41f8-b9ea-a922c5c8bf7e-kube-api-access-ghqfk\") pod \"node-exporter-ddq2n\" (UID: \"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e\") " pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.301388 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.301304 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ddq2n" Jul 08 18:22:58.310473 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:22:58.310443 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05245de8_9d7d_41f8_b9ea_a922c5c8bf7e.slice/crio-2091b6b3af03acdacd38f5441bda33fa6ef7a6870afd0183a697d574216f8637 WatchSource:0}: Error finding container 2091b6b3af03acdacd38f5441bda33fa6ef7a6870afd0183a697d574216f8637: Status 404 returned error can't find the container with id 2091b6b3af03acdacd38f5441bda33fa6ef7a6870afd0183a697d574216f8637 Jul 08 18:22:58.340468 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:58.340418 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddq2n" event={"ID":"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e","Type":"ContainerStarted","Data":"2091b6b3af03acdacd38f5441bda33fa6ef7a6870afd0183a697d574216f8637"} Jul 08 18:22:59.866815 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:22:59.866711 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" podUID="e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 18:23:00.323902 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:23:00.323858 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cgqjt" podUID="31aa72aa-4b52-4112-a90a-f8b9f4784008" Jul 08 18:23:00.347988 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:00.347956 2562 generic.go:356] "Generic (PLEG): container finished" podID="05245de8-9d7d-41f8-b9ea-a922c5c8bf7e" containerID="5c4076ba30855d0f0857af1a93209c9cf87413377bf36ff02a984c4b05c75732" exitCode=0 Jul 08 18:23:00.348151 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:00.348017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddq2n" event={"ID":"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e","Type":"ContainerDied","Data":"5c4076ba30855d0f0857af1a93209c9cf87413377bf36ff02a984c4b05c75732"} Jul 08 18:23:01.353057 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:01.353020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddq2n" event={"ID":"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e","Type":"ContainerStarted","Data":"008ad807aa142d42d914a3f0f620c67e82bb59c4bdbced00a55855a9b8c37a73"} Jul 08 18:23:01.353455 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:01.353072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ddq2n" event={"ID":"05245de8-9d7d-41f8-b9ea-a922c5c8bf7e","Type":"ContainerStarted","Data":"a818c53228d8bca88b9c4bc6b2654863a441a81bd56c97005e9a6ea7d732c860"} Jul 08 18:23:01.380377 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:01.380319 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ddq2n" podStartSLOduration=3.127902364 podStartE2EDuration="4.380302094s" totalImagesPullingTime="1.181572526s" totalInitContainerRuntime="70.827204ms" isStatefulPod=true podCreationTimestamp="2026-07-08 18:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:01.379506756 +0000 UTC m=+294.641572508" watchObservedRunningTime="2026-07-08 18:23:01.380302094 +0000 UTC m=+294.642367844" Jul 08 18:23:02.403665 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.403628 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5687887474-m2vgb"] Jul 08 18:23:02.405835 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.405818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.408294 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.408220 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 18:23:02.408475 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.408460 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 18:23:02.408546 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.408466 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9ljj4lcsut7us\"" Jul 08 18:23:02.409421 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.409397 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 18:23:02.409495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.409435 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 18:23:02.409549 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.409520 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2t5sh\"" Jul 08 18:23:02.415859 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.415840 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5687887474-m2vgb"] Jul 08 18:23:02.423478 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb671a63-0697-485a-a177-01c3475259d2-audit-log\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.423599 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423489 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb671a63-0697-485a-a177-01c3475259d2-metrics-server-audit-profiles\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.423599 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-secret-metrics-server-tls\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.423701 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb671a63-0697-485a-a177-01c3475259d2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.423701 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-secret-metrics-server-client-certs\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.423890 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423823 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-client-ca-bundle\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.424017 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.423901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr22q\" (UniqueName: \"kubernetes.io/projected/cb671a63-0697-485a-a177-01c3475259d2-kube-api-access-kr22q\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.524834 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.524794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-client-ca-bundle\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.524834 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.524838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr22q\" (UniqueName: \"kubernetes.io/projected/cb671a63-0697-485a-a177-01c3475259d2-kube-api-access-kr22q\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525068 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.524868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb671a63-0697-485a-a177-01c3475259d2-audit-log\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525068 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.524883 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb671a63-0697-485a-a177-01c3475259d2-metrics-server-audit-profiles\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525068 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.525005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-secret-metrics-server-tls\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525068 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.525053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb671a63-0697-485a-a177-01c3475259d2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525249 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.525104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-secret-metrics-server-client-certs\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525489 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.525450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb671a63-0697-485a-a177-01c3475259d2-audit-log\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.525759 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.525734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb671a63-0697-485a-a177-01c3475259d2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.526169 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.526144 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb671a63-0697-485a-a177-01c3475259d2-metrics-server-audit-profiles\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.527431 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.527413 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-client-ca-bundle\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.527584 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.527568 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-secret-metrics-server-client-certs\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.527628 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.527593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb671a63-0697-485a-a177-01c3475259d2-secret-metrics-server-tls\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.534212 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.534190 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr22q\" (UniqueName: \"kubernetes.io/projected/cb671a63-0697-485a-a177-01c3475259d2-kube-api-access-kr22q\") pod \"metrics-server-5687887474-m2vgb\" (UID: \"cb671a63-0697-485a-a177-01c3475259d2\") " pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.714964 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.714884 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:02.840055 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:02.840027 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5687887474-m2vgb"] Jul 08 18:23:02.843761 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:23:02.843691 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb671a63_0697_485a_a177_01c3475259d2.slice/crio-9f2e2fd0a4dfb5c83381ecbb3799a9ba5c0b8211efceae006e1792e0c2373e92 WatchSource:0}: Error finding container 9f2e2fd0a4dfb5c83381ecbb3799a9ba5c0b8211efceae006e1792e0c2373e92: Status 404 returned error can't find the container with id 9f2e2fd0a4dfb5c83381ecbb3799a9ba5c0b8211efceae006e1792e0c2373e92 Jul 08 18:23:03.363954 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:03.363911 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" event={"ID":"cb671a63-0697-485a-a177-01c3475259d2","Type":"ContainerStarted","Data":"9f2e2fd0a4dfb5c83381ecbb3799a9ba5c0b8211efceae006e1792e0c2373e92"} Jul 08 18:23:04.369006 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:04.368967 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" event={"ID":"cb671a63-0697-485a-a177-01c3475259d2","Type":"ContainerStarted","Data":"57cb9a0735e575d216ef56658ef9ef554f362a3320d9e63c5b1fe359cc53f72a"} Jul 08 18:23:04.386371 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:04.386321 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" podStartSLOduration=0.969991263 podStartE2EDuration="2.386305074s" totalImagesPullingTime="1.416313811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:23:04.385128497 +0000 UTC m=+297.647194261" watchObservedRunningTime="2026-07-08 18:23:04.386305074 +0000 UTC m=+297.648370825" Jul 08 18:23:05.329036 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:05.329009 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5gl29" Jul 08 18:23:07.138627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:07.138596 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 18:23:09.867048 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:09.867007 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" podUID="e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 18:23:12.322179 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:12.322138 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:23:19.866305 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:19.866245 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" podUID="e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 18:23:19.866655 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:19.866340 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" Jul 08 18:23:19.866885 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:19.866855 2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"f9d7d64b904e1f52b1bed740d34df62add725d47dce3e8d5806716ad0c8b5fc1"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 08 18:23:19.866925 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:19.866909 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" podUID="e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872" containerName="service-proxy" containerID="cri-o://f9d7d64b904e1f52b1bed740d34df62add725d47dce3e8d5806716ad0c8b5fc1" gracePeriod=30 Jul 08 18:23:20.419354 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:20.419321 2562 generic.go:356] "Generic (PLEG): container finished" podID="e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872" containerID="f9d7d64b904e1f52b1bed740d34df62add725d47dce3e8d5806716ad0c8b5fc1" exitCode=2 Jul 08 18:23:20.419531 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:20.419390 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" event={"ID":"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872","Type":"ContainerDied","Data":"f9d7d64b904e1f52b1bed740d34df62add725d47dce3e8d5806716ad0c8b5fc1"} Jul 08 18:23:20.419531 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:20.419436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558fc8966b-g87ft" event={"ID":"e68f0ed0-48aa-4f8b-b7e9-4f9b4751f872","Type":"ContainerStarted","Data":"5013baf86971e86faf202f03c84c3620df5ba67fbecf9f07f283b8fd14f020c7"} Jul 08 18:23:22.715110 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:22.715063 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:22.715599 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:22.715138 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:42.721139 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:42.721107 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:23:42.724972 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:23:42.724950 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5687887474-m2vgb" Jul 08 18:24:20.103715 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.103620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:24:20.105965 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.105945 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31aa72aa-4b52-4112-a90a-f8b9f4784008-metrics-certs\") pod \"network-metrics-daemon-cgqjt\" (UID: \"31aa72aa-4b52-4112-a90a-f8b9f4784008\") " pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:24:20.125726 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.125697 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9qsc4\"" Jul 08 18:24:20.133647 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.133630 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgqjt" Jul 08 18:24:20.254133 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.254109 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cgqjt"] Jul 08 18:24:20.256583 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:24:20.256557 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31aa72aa_4b52_4112_a90a_f8b9f4784008.slice/crio-5bde15abb35ba777287c3506fb92099839d9027ee77ca8b48cfed9478169e043 WatchSource:0}: Error finding container 5bde15abb35ba777287c3506fb92099839d9027ee77ca8b48cfed9478169e043: Status 404 returned error can't find the container with id 5bde15abb35ba777287c3506fb92099839d9027ee77ca8b48cfed9478169e043 Jul 08 18:24:20.258636 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.258621 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:24:20.603628 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:20.603592 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgqjt" event={"ID":"31aa72aa-4b52-4112-a90a-f8b9f4784008","Type":"ContainerStarted","Data":"5bde15abb35ba777287c3506fb92099839d9027ee77ca8b48cfed9478169e043"} Jul 08 18:24:21.608612 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:21.608512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgqjt" event={"ID":"31aa72aa-4b52-4112-a90a-f8b9f4784008","Type":"ContainerStarted","Data":"8927a2e768a814ed8e0217bc6ca080256de3a90a4709fc6c57026b0292cca683"} Jul 08 18:24:21.609014 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:21.608612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgqjt" event={"ID":"31aa72aa-4b52-4112-a90a-f8b9f4784008","Type":"ContainerStarted","Data":"8d4ff061f9d766b3c1c3b9cef9a8535d7a7b67c68352052369be3f2a1dfb0ec5"} Jul 08 18:24:21.625179 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:24:21.625135 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cgqjt" podStartSLOduration=373.540457286 podStartE2EDuration="6m14.625121636s" totalImagesPullingTime="1.08466435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:24:21.623984725 +0000 UTC m=+374.886050476" watchObservedRunningTime="2026-07-08 18:24:21.625121636 +0000 UTC m=+374.887187386" Jul 08 18:25:55.223880 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.223845 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844"] Jul 08 18:25:55.226202 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.226186 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.228674 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.228647 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:25:55.228790 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.228647 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jlt8z\"" Jul 08 18:25:55.228790 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.228739 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 18:25:55.239518 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.239494 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844"] Jul 08 18:25:55.263587 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.263552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1b44084-38ae-4d72-affa-e0f697338c85-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tj844\" (UID: \"b1b44084-38ae-4d72-affa-e0f697338c85\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.263731 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.263639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx5jp\" (UniqueName: \"kubernetes.io/projected/b1b44084-38ae-4d72-affa-e0f697338c85-kube-api-access-gx5jp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tj844\" (UID: \"b1b44084-38ae-4d72-affa-e0f697338c85\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.364561 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.364523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1b44084-38ae-4d72-affa-e0f697338c85-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tj844\" (UID: \"b1b44084-38ae-4d72-affa-e0f697338c85\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.364762 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.364602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx5jp\" (UniqueName: \"kubernetes.io/projected/b1b44084-38ae-4d72-affa-e0f697338c85-kube-api-access-gx5jp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tj844\" (UID: \"b1b44084-38ae-4d72-affa-e0f697338c85\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.364956 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.364935 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b1b44084-38ae-4d72-affa-e0f697338c85-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tj844\" (UID: \"b1b44084-38ae-4d72-affa-e0f697338c85\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.376022 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.375995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx5jp\" (UniqueName: \"kubernetes.io/projected/b1b44084-38ae-4d72-affa-e0f697338c85-kube-api-access-gx5jp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tj844\" (UID: \"b1b44084-38ae-4d72-affa-e0f697338c85\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.534604 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.534515 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" Jul 08 18:25:55.658451 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.658427 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844"] Jul 08 18:25:55.660785 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:25:55.660758 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b44084_38ae_4d72_affa_e0f697338c85.slice/crio-26cb28816c0d50ae0a5f37b5004e677e0c449d16e07b911d9b609df9ce2dedf5 WatchSource:0}: Error finding container 26cb28816c0d50ae0a5f37b5004e677e0c449d16e07b911d9b609df9ce2dedf5: Status 404 returned error can't find the container with id 26cb28816c0d50ae0a5f37b5004e677e0c449d16e07b911d9b609df9ce2dedf5 Jul 08 18:25:55.917333 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:55.917228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" event={"ID":"b1b44084-38ae-4d72-affa-e0f697338c85","Type":"ContainerStarted","Data":"26cb28816c0d50ae0a5f37b5004e677e0c449d16e07b911d9b609df9ce2dedf5"} Jul 08 18:25:58.929512 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:58.929474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" event={"ID":"b1b44084-38ae-4d72-affa-e0f697338c85","Type":"ContainerStarted","Data":"d47656e9fe6d33ebdce05d49319bc8d35e49d11837a8b4b559002e329b9b6271"} Jul 08 18:25:58.953223 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:25:58.953174 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tj844" podStartSLOduration=1.4806071219999999 podStartE2EDuration="3.953156133s" totalImagesPullingTime="2.472549011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:25:58.951244464 +0000 UTC m=+472.213310208" watchObservedRunningTime="2026-07-08 18:25:58.953156133 +0000 UTC m=+472.215221883" Jul 08 18:26:11.629551 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.629520 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jdp9q"] Jul 08 18:26:11.631610 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.631591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.633881 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.633851 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 18:26:11.634859 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.634841 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-tbl65\"" Jul 08 18:26:11.634947 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.634885 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 18:26:11.644679 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.644656 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jdp9q"] Jul 08 18:26:11.678998 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.678959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6psf\" (UniqueName: \"kubernetes.io/projected/62f73357-b895-4c2b-b1b8-5f6facde4eef-kube-api-access-g6psf\") pod \"cert-manager-55b4d954d5-jdp9q\" (UID: \"62f73357-b895-4c2b-b1b8-5f6facde4eef\") " pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.679169 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.679025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62f73357-b895-4c2b-b1b8-5f6facde4eef-bound-sa-token\") pod \"cert-manager-55b4d954d5-jdp9q\" (UID: \"62f73357-b895-4c2b-b1b8-5f6facde4eef\") " pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.779977 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.779935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62f73357-b895-4c2b-b1b8-5f6facde4eef-bound-sa-token\") pod \"cert-manager-55b4d954d5-jdp9q\" (UID: \"62f73357-b895-4c2b-b1b8-5f6facde4eef\") " pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.780135 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.780003 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6psf\" (UniqueName: \"kubernetes.io/projected/62f73357-b895-4c2b-b1b8-5f6facde4eef-kube-api-access-g6psf\") pod \"cert-manager-55b4d954d5-jdp9q\" (UID: \"62f73357-b895-4c2b-b1b8-5f6facde4eef\") " pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.787915 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.787876 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/62f73357-b895-4c2b-b1b8-5f6facde4eef-bound-sa-token\") pod \"cert-manager-55b4d954d5-jdp9q\" (UID: \"62f73357-b895-4c2b-b1b8-5f6facde4eef\") " pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.788034 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.787970 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6psf\" (UniqueName: \"kubernetes.io/projected/62f73357-b895-4c2b-b1b8-5f6facde4eef-kube-api-access-g6psf\") pod \"cert-manager-55b4d954d5-jdp9q\" (UID: \"62f73357-b895-4c2b-b1b8-5f6facde4eef\") " pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:11.940344 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:11.940246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jdp9q" Jul 08 18:26:12.069493 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:12.069461 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jdp9q"] Jul 08 18:26:12.072251 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:26:12.072216 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62f73357_b895_4c2b_b1b8_5f6facde4eef.slice/crio-1fd7637c654803f2c6dda7c799288efa6f18bfb5c383655328fe059274245f33 WatchSource:0}: Error finding container 1fd7637c654803f2c6dda7c799288efa6f18bfb5c383655328fe059274245f33: Status 404 returned error can't find the container with id 1fd7637c654803f2c6dda7c799288efa6f18bfb5c383655328fe059274245f33 Jul 08 18:26:12.979575 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:12.979538 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jdp9q" event={"ID":"62f73357-b895-4c2b-b1b8-5f6facde4eef","Type":"ContainerStarted","Data":"1fd7637c654803f2c6dda7c799288efa6f18bfb5c383655328fe059274245f33"} Jul 08 18:26:14.990177 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:14.990139 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jdp9q" event={"ID":"62f73357-b895-4c2b-b1b8-5f6facde4eef","Type":"ContainerStarted","Data":"05c3f2e166949b340a2c60121a61b250f8baafdc8c32448228e91d6cd7c510ad"} Jul 08 18:26:15.013338 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:15.013287 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jdp9q" podStartSLOduration=1.634238137 podStartE2EDuration="4.013248013s" totalImagesPullingTime="2.379009876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:26:15.012441895 +0000 UTC m=+488.274507645" watchObservedRunningTime="2026-07-08 18:26:15.013248013 +0000 UTC m=+488.275313760" Jul 08 18:26:36.290598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.290566 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d"] Jul 08 18:26:36.295285 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.295240 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.297734 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.297718 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q6glk\"" Jul 08 18:26:36.297918 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.297906 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 18:26:36.298510 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.298411 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 18:26:36.298510 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.298423 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 18:26:36.298510 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.298430 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 18:26:36.308314 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.308290 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d"] Jul 08 18:26:36.365821 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.365787 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87cde5bd-973f-444b-974c-42ddacccd5e9-tmp\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.365821 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.365824 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/87cde5bd-973f-444b-974c-42ddacccd5e9-manifests\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.366047 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.365886 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87cde5bd-973f-444b-974c-42ddacccd5e9-webhook-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.366047 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.365923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/87cde5bd-973f-444b-974c-42ddacccd5e9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.366047 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.365946 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9rp\" (UniqueName: \"kubernetes.io/projected/87cde5bd-973f-444b-974c-42ddacccd5e9-kube-api-access-kx9rp\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.467247 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.467202 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87cde5bd-973f-444b-974c-42ddacccd5e9-webhook-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.467247 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.467254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/87cde5bd-973f-444b-974c-42ddacccd5e9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.467532 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.467305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9rp\" (UniqueName: \"kubernetes.io/projected/87cde5bd-973f-444b-974c-42ddacccd5e9-kube-api-access-kx9rp\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.467532 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.467345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87cde5bd-973f-444b-974c-42ddacccd5e9-tmp\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.467532 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.467371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/87cde5bd-973f-444b-974c-42ddacccd5e9-manifests\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.467799 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.467781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/87cde5bd-973f-444b-974c-42ddacccd5e9-manifests\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.469709 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.469685 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/87cde5bd-973f-444b-974c-42ddacccd5e9-tmp\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.469820 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.469725 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/87cde5bd-973f-444b-974c-42ddacccd5e9-webhook-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.469876 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.469845 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/87cde5bd-973f-444b-974c-42ddacccd5e9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.476156 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.476129 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9rp\" (UniqueName: \"kubernetes.io/projected/87cde5bd-973f-444b-974c-42ddacccd5e9-kube-api-access-kx9rp\") pod \"opendatahub-operator-controller-manager-578597fb9c-zjf4d\" (UID: \"87cde5bd-973f-444b-974c-42ddacccd5e9\") " pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.605608 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.605523 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:36.739929 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:36.739898 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d"] Jul 08 18:26:36.743661 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:26:36.743634 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87cde5bd_973f_444b_974c_42ddacccd5e9.slice/crio-fe06dffd8aa69cdc227484e963384efbb4dd97224a90a21512a39fe2ffcafed4 WatchSource:0}: Error finding container fe06dffd8aa69cdc227484e963384efbb4dd97224a90a21512a39fe2ffcafed4: Status 404 returned error can't find the container with id fe06dffd8aa69cdc227484e963384efbb4dd97224a90a21512a39fe2ffcafed4 Jul 08 18:26:37.064477 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:37.064442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" event={"ID":"87cde5bd-973f-444b-974c-42ddacccd5e9","Type":"ContainerStarted","Data":"fe06dffd8aa69cdc227484e963384efbb4dd97224a90a21512a39fe2ffcafed4"} Jul 08 18:26:40.079404 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:40.079360 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" event={"ID":"87cde5bd-973f-444b-974c-42ddacccd5e9","Type":"ContainerStarted","Data":"1fa1058776329e65af14d6d4b7f5fc98dd671eef0be1072d11c4733f95382f2a"} Jul 08 18:26:41.084782 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:41.084746 2562 generic.go:356] "Generic (PLEG): container finished" podID="87cde5bd-973f-444b-974c-42ddacccd5e9" containerID="1fa1058776329e65af14d6d4b7f5fc98dd671eef0be1072d11c4733f95382f2a" exitCode=0 Jul 08 18:26:41.085180 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:41.084812 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" event={"ID":"87cde5bd-973f-444b-974c-42ddacccd5e9","Type":"ContainerDied","Data":"1fa1058776329e65af14d6d4b7f5fc98dd671eef0be1072d11c4733f95382f2a"} Jul 08 18:26:42.090517 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:42.090481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" event={"ID":"87cde5bd-973f-444b-974c-42ddacccd5e9","Type":"ContainerStarted","Data":"52501f93af263f07227a14314ed77c331ec2fa3addf74729b9eeed26809861d0"} Jul 08 18:26:42.090916 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:42.090631 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:42.112134 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:42.112065 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" podStartSLOduration=1.80455392 podStartE2EDuration="6.112049743s" totalImagesPullingTime="3.307435856s" totalInitContainerRuntime="1.000059967s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:26:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:26:42.110519808 +0000 UTC m=+515.372585574" watchObservedRunningTime="2026-07-08 18:26:42.112049743 +0000 UTC m=+515.374115493" Jul 08 18:26:53.096585 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:53.096552 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-578597fb9c-zjf4d" Jul 08 18:26:55.759670 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.759635 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt"] Jul 08 18:26:55.765560 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.765539 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:55.768456 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.768426 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 18:26:55.768456 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.768447 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 18:26:55.769511 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.769496 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 18:26:55.769577 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.769498 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 18:26:55.769637 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.769620 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zk85b\"" Jul 08 18:26:55.785574 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.785546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt"] Jul 08 18:26:55.803942 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.803911 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5"] Jul 08 18:26:55.806575 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.806555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:55.809856 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.809839 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 18:26:55.810085 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.810059 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 18:26:55.810391 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.810354 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k578t\"" Jul 08 18:26:55.810465 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.810428 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 18:26:55.810552 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.810534 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 18:26:55.810627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.810609 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 18:26:55.819628 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.819606 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5"] Jul 08 18:26:55.915181 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-manager-config\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:55.915181 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915184 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rh2c\" (UniqueName: \"kubernetes.io/projected/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-kube-api-access-4rh2c\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:55.915425 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-tls-certs\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:55.915425 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915234 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnvdq\" (UniqueName: \"kubernetes.io/projected/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-kube-api-access-vnvdq\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:55.915425 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-metrics-cert\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:55.915425 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-cert\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:55.915425 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:55.915386 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-tmp\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.016720 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-cert\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.016720 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-tmp\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.016720 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-manager-config\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.016720 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016715 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rh2c\" (UniqueName: \"kubernetes.io/projected/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-kube-api-access-4rh2c\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.017045 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-tls-certs\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.017045 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnvdq\" (UniqueName: \"kubernetes.io/projected/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-kube-api-access-vnvdq\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.017045 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.016787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-metrics-cert\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.017508 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.017470 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-manager-config\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.019294 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.019251 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-tls-certs\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.019381 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.019278 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-metrics-cert\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.019381 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.019350 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-tmp\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.019680 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.019661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-cert\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.026213 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.026190 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnvdq\" (UniqueName: \"kubernetes.io/projected/34c02218-ccd5-4ba9-9ce7-4b971e963fcf-kube-api-access-vnvdq\") pod \"lws-controller-manager-56665c6f48-9nqb5\" (UID: \"34c02218-ccd5-4ba9-9ce7-4b971e963fcf\") " pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.026351 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.026324 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rh2c\" (UniqueName: \"kubernetes.io/projected/ce0dcf26-a30f-4c4c-99b0-bd5693a87118-kube-api-access-4rh2c\") pod \"kube-auth-proxy-5fb94956b8-l5brt\" (UID: \"ce0dcf26-a30f-4c4c-99b0-bd5693a87118\") " pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.075337 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.075310 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" Jul 08 18:26:56.115225 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.115193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:26:56.213069 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.213033 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt"] Jul 08 18:26:56.217042 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:26:56.217009 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0dcf26_a30f_4c4c_99b0_bd5693a87118.slice/crio-b3436101ec0c7ab510f7730787d28e3f468dd271accbb34fe07723a050a19e3a WatchSource:0}: Error finding container b3436101ec0c7ab510f7730787d28e3f468dd271accbb34fe07723a050a19e3a: Status 404 returned error can't find the container with id b3436101ec0c7ab510f7730787d28e3f468dd271accbb34fe07723a050a19e3a Jul 08 18:26:56.269064 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:56.269044 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5"] Jul 08 18:26:56.271056 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:26:56.271028 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c02218_ccd5_4ba9_9ce7_4b971e963fcf.slice/crio-f29cdfcb0f5d3bcab1fccbbc4a1a61097ab310ffb8e706db8a1116b3c666e29b WatchSource:0}: Error finding container f29cdfcb0f5d3bcab1fccbbc4a1a61097ab310ffb8e706db8a1116b3c666e29b: Status 404 returned error can't find the container with id f29cdfcb0f5d3bcab1fccbbc4a1a61097ab310ffb8e706db8a1116b3c666e29b Jul 08 18:26:57.150561 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:57.150474 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" event={"ID":"34c02218-ccd5-4ba9-9ce7-4b971e963fcf","Type":"ContainerStarted","Data":"f29cdfcb0f5d3bcab1fccbbc4a1a61097ab310ffb8e706db8a1116b3c666e29b"} Jul 08 18:26:57.151848 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:26:57.151819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" event={"ID":"ce0dcf26-a30f-4c4c-99b0-bd5693a87118","Type":"ContainerStarted","Data":"b3436101ec0c7ab510f7730787d28e3f468dd271accbb34fe07723a050a19e3a"} Jul 08 18:27:00.166379 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:27:00.166340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" event={"ID":"ce0dcf26-a30f-4c4c-99b0-bd5693a87118","Type":"ContainerStarted","Data":"eb0025c666773cd8a8ea62501f5fe8a10e3a017f71842cf716104e02d257c9a3"} Jul 08 18:27:00.167764 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:27:00.167742 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" event={"ID":"34c02218-ccd5-4ba9-9ce7-4b971e963fcf","Type":"ContainerStarted","Data":"f810760907f95c6b6af624b088fe0e0da909ef0789b9bedae0dc5cbefaeedf96"} Jul 08 18:27:00.167879 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:27:00.167865 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:27:00.185099 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:27:00.185056 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fb94956b8-l5brt" podStartSLOduration=1.9383151889999999 podStartE2EDuration="5.185019267s" totalImagesPullingTime="3.246704078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:27:00.184311223 +0000 UTC m=+533.446376974" watchObservedRunningTime="2026-07-08 18:27:00.185019267 +0000 UTC m=+533.447085018" Jul 08 18:27:00.200974 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:27:00.200918 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" podStartSLOduration=2.009428198 podStartE2EDuration="5.200898797s" totalImagesPullingTime="3.191470599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:27:00.200409588 +0000 UTC m=+533.462475340" watchObservedRunningTime="2026-07-08 18:27:00.200898797 +0000 UTC m=+533.462964548" Jul 08 18:27:11.173838 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:27:11.173809 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56665c6f48-9nqb5" Jul 08 18:28:38.603958 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.603886 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb"] Jul 08 18:28:38.605866 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.605851 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.609082 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.609064 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 18:28:38.609180 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.609096 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kg9gr\"" Jul 08 18:28:38.609788 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.609766 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 18:28:38.609788 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.609771 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 18:28:38.609925 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.609821 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 18:28:38.616031 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.616011 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb"] Jul 08 18:28:38.672325 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.672291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/645754bc-1a1b-4dce-843a-1634b539e041-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.672502 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.672346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/645754bc-1a1b-4dce-843a-1634b539e041-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.672502 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.672405 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2c5w\" (UniqueName: \"kubernetes.io/projected/645754bc-1a1b-4dce-843a-1634b539e041-kube-api-access-k2c5w\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.773404 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.773365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/645754bc-1a1b-4dce-843a-1634b539e041-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.773585 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.773418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/645754bc-1a1b-4dce-843a-1634b539e041-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.773585 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.773459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2c5w\" (UniqueName: \"kubernetes.io/projected/645754bc-1a1b-4dce-843a-1634b539e041-kube-api-access-k2c5w\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.774014 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.773990 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/645754bc-1a1b-4dce-843a-1634b539e041-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.775978 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.775962 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/645754bc-1a1b-4dce-843a-1634b539e041-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.787601 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.787576 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2c5w\" (UniqueName: \"kubernetes.io/projected/645754bc-1a1b-4dce-843a-1634b539e041-kube-api-access-k2c5w\") pod \"kuadrant-console-plugin-66b5774985-5xlpb\" (UID: \"645754bc-1a1b-4dce-843a-1634b539e041\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:38.914376 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:38.914249 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" Jul 08 18:28:39.041528 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:39.041498 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb"] Jul 08 18:28:39.043884 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:28:39.043856 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod645754bc_1a1b_4dce_843a_1634b539e041.slice/crio-35c4103263b6d00786d6882d8549c7df535e514ca06b7a70d85b45cb18716912 WatchSource:0}: Error finding container 35c4103263b6d00786d6882d8549c7df535e514ca06b7a70d85b45cb18716912: Status 404 returned error can't find the container with id 35c4103263b6d00786d6882d8549c7df535e514ca06b7a70d85b45cb18716912 Jul 08 18:28:39.548099 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:28:39.548061 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" event={"ID":"645754bc-1a1b-4dce-843a-1634b539e041","Type":"ContainerStarted","Data":"35c4103263b6d00786d6882d8549c7df535e514ca06b7a70d85b45cb18716912"} Jul 08 18:29:01.649287 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:01.649233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" event={"ID":"645754bc-1a1b-4dce-843a-1634b539e041","Type":"ContainerStarted","Data":"5cf971cebbae5aff742c65fba2156ecf534096f388f105790cb2975e5b069f60"} Jul 08 18:29:01.665439 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:01.665393 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5xlpb" podStartSLOduration=1.845823271 podStartE2EDuration="23.665377846s" totalImagesPullingTime="21.819554575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:29:01.66455534 +0000 UTC m=+654.926621091" watchObservedRunningTime="2026-07-08 18:29:01.665377846 +0000 UTC m=+654.927443651" Jul 08 18:29:23.503182 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.503145 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:29:23.550790 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.550764 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:29:23.550790 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.550798 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:29:23.550982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.550884 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.554041 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.554021 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 18:29:23.748855 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.748816 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtgd5\" (UniqueName: \"kubernetes.io/projected/89180bfa-5115-4a51-bfb7-056d5e5292b1-kube-api-access-xtgd5\") pod \"limitador-limitador-54d6d988d5-8n2h8\" (UID: \"89180bfa-5115-4a51-bfb7-056d5e5292b1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.748855 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.748862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89180bfa-5115-4a51-bfb7-056d5e5292b1-config-file\") pod \"limitador-limitador-54d6d988d5-8n2h8\" (UID: \"89180bfa-5115-4a51-bfb7-056d5e5292b1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.849862 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.849778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89180bfa-5115-4a51-bfb7-056d5e5292b1-config-file\") pod \"limitador-limitador-54d6d988d5-8n2h8\" (UID: \"89180bfa-5115-4a51-bfb7-056d5e5292b1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.850005 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.849871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtgd5\" (UniqueName: \"kubernetes.io/projected/89180bfa-5115-4a51-bfb7-056d5e5292b1-kube-api-access-xtgd5\") pod \"limitador-limitador-54d6d988d5-8n2h8\" (UID: \"89180bfa-5115-4a51-bfb7-056d5e5292b1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.850456 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.850438 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/89180bfa-5115-4a51-bfb7-056d5e5292b1-config-file\") pod \"limitador-limitador-54d6d988d5-8n2h8\" (UID: \"89180bfa-5115-4a51-bfb7-056d5e5292b1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.858293 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.858236 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtgd5\" (UniqueName: \"kubernetes.io/projected/89180bfa-5115-4a51-bfb7-056d5e5292b1-kube-api-access-xtgd5\") pod \"limitador-limitador-54d6d988d5-8n2h8\" (UID: \"89180bfa-5115-4a51-bfb7-056d5e5292b1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.859996 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.859981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:23.936507 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.936475 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-wmmw5"] Jul 08 18:29:23.940066 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.939822 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:23.942473 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.942426 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t5ps7\"" Jul 08 18:29:23.948677 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.948596 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wmmw5"] Jul 08 18:29:23.991663 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.991628 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:29:23.994421 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:29:23.994391 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89180bfa_5115_4a51_bfb7_056d5e5292b1.slice/crio-0429c8d5e7c1d1de06b7a2b0e2793001fd69a5b6847da9f6cbda549b4d69e0f5 WatchSource:0}: Error finding container 0429c8d5e7c1d1de06b7a2b0e2793001fd69a5b6847da9f6cbda549b4d69e0f5: Status 404 returned error can't find the container with id 0429c8d5e7c1d1de06b7a2b0e2793001fd69a5b6847da9f6cbda549b4d69e0f5 Jul 08 18:29:23.996464 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:23.996447 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:29:24.051556 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.051520 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xqc\" (UniqueName: \"kubernetes.io/projected/c0da055f-fe92-4ff0-94c4-34173ba81de6-kube-api-access-g2xqc\") pod \"authorino-58cd955675-wmmw5\" (UID: \"c0da055f-fe92-4ff0-94c4-34173ba81de6\") " pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:24.152658 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.152572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xqc\" (UniqueName: \"kubernetes.io/projected/c0da055f-fe92-4ff0-94c4-34173ba81de6-kube-api-access-g2xqc\") pod \"authorino-58cd955675-wmmw5\" (UID: \"c0da055f-fe92-4ff0-94c4-34173ba81de6\") " pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:24.160836 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.160792 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xqc\" (UniqueName: \"kubernetes.io/projected/c0da055f-fe92-4ff0-94c4-34173ba81de6-kube-api-access-g2xqc\") pod \"authorino-58cd955675-wmmw5\" (UID: \"c0da055f-fe92-4ff0-94c4-34173ba81de6\") " pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:24.253245 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.253203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:24.376314 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.376253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wmmw5"] Jul 08 18:29:24.379828 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:29:24.379797 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0da055f_fe92_4ff0_94c4_34173ba81de6.slice/crio-335b24006f20738a0724811e8a88b7d84233b72290610795f978b1f560a39151 WatchSource:0}: Error finding container 335b24006f20738a0724811e8a88b7d84233b72290610795f978b1f560a39151: Status 404 returned error can't find the container with id 335b24006f20738a0724811e8a88b7d84233b72290610795f978b1f560a39151 Jul 08 18:29:24.739466 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.739374 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" event={"ID":"89180bfa-5115-4a51-bfb7-056d5e5292b1","Type":"ContainerStarted","Data":"0429c8d5e7c1d1de06b7a2b0e2793001fd69a5b6847da9f6cbda549b4d69e0f5"} Jul 08 18:29:24.741602 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:24.741570 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wmmw5" event={"ID":"c0da055f-fe92-4ff0-94c4-34173ba81de6","Type":"ContainerStarted","Data":"335b24006f20738a0724811e8a88b7d84233b72290610795f978b1f560a39151"} Jul 08 18:29:28.763545 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:28.763509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" event={"ID":"89180bfa-5115-4a51-bfb7-056d5e5292b1","Type":"ContainerStarted","Data":"b3fd85b0bb57a2665e0722b63e52449a3df316e164db1d9a7b003b80697c0c3d"} Jul 08 18:29:28.763980 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:28.763588 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:28.764929 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:28.764905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wmmw5" event={"ID":"c0da055f-fe92-4ff0-94c4-34173ba81de6","Type":"ContainerStarted","Data":"44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2"} Jul 08 18:29:28.781439 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:28.781397 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" podStartSLOduration=1.460863599 podStartE2EDuration="5.781382149s" totalImagesPullingTime="4.32051855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:29:28.779368911 +0000 UTC m=+682.041434661" watchObservedRunningTime="2026-07-08 18:29:28.781382149 +0000 UTC m=+682.043447900" Jul 08 18:29:28.795349 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:28.795297 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-wmmw5" podStartSLOduration=1.8165392489999999 podStartE2EDuration="5.795256136s" totalImagesPullingTime="3.978716887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:29:28.794080573 +0000 UTC m=+682.056146339" watchObservedRunningTime="2026-07-08 18:29:28.795256136 +0000 UTC m=+682.057321886" Jul 08 18:29:29.133227 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:29.133144 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wmmw5"] Jul 08 18:29:30.773396 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:30.773340 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-wmmw5" podUID="c0da055f-fe92-4ff0-94c4-34173ba81de6" containerName="authorino" containerID="cri-o://44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2" gracePeriod=30 Jul 08 18:29:31.059073 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.058986 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:31.103098 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.103062 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2xqc\" (UniqueName: \"kubernetes.io/projected/c0da055f-fe92-4ff0-94c4-34173ba81de6-kube-api-access-g2xqc\") pod \"c0da055f-fe92-4ff0-94c4-34173ba81de6\" (UID: \"c0da055f-fe92-4ff0-94c4-34173ba81de6\") " Jul 08 18:29:31.105477 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.105455 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0da055f-fe92-4ff0-94c4-34173ba81de6-kube-api-access-g2xqc" (OuterVolumeSpecName: "kube-api-access-g2xqc") pod "c0da055f-fe92-4ff0-94c4-34173ba81de6" (UID: "c0da055f-fe92-4ff0-94c4-34173ba81de6"). InnerVolumeSpecName "kube-api-access-g2xqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:29:31.204311 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.204258 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2xqc\" (UniqueName: \"kubernetes.io/projected/c0da055f-fe92-4ff0-94c4-34173ba81de6-kube-api-access-g2xqc\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:29:31.777850 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.777813 2562 generic.go:356] "Generic (PLEG): container finished" podID="c0da055f-fe92-4ff0-94c4-34173ba81de6" containerID="44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2" exitCode=0 Jul 08 18:29:31.777850 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.777854 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wmmw5" Jul 08 18:29:31.778317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.777901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wmmw5" event={"ID":"c0da055f-fe92-4ff0-94c4-34173ba81de6","Type":"ContainerDied","Data":"44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2"} Jul 08 18:29:31.778317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.777939 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wmmw5" event={"ID":"c0da055f-fe92-4ff0-94c4-34173ba81de6","Type":"ContainerDied","Data":"335b24006f20738a0724811e8a88b7d84233b72290610795f978b1f560a39151"} Jul 08 18:29:31.778317 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.777967 2562 scope.go:117] "RemoveContainer" containerID="44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2" Jul 08 18:29:31.786194 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.786176 2562 scope.go:117] "RemoveContainer" containerID="44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2" Jul 08 18:29:31.786489 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:29:31.786461 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2\": container with ID starting with 44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2 not found: ID does not exist" containerID="44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2" Jul 08 18:29:31.786546 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.786493 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2"} err="failed to get container status \"44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2\": rpc error: code = NotFound desc = could not find container \"44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2\": container with ID starting with 44930ce9ba3d186f8d621f4bba4be4aa17843f12472734e8aab4f746023fa0f2 not found: ID does not exist" Jul 08 18:29:31.792467 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.792445 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wmmw5"] Jul 08 18:29:31.797728 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:31.797710 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-wmmw5"] Jul 08 18:29:33.326063 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:33.326031 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0da055f-fe92-4ff0-94c4-34173ba81de6" path="/var/lib/kubelet/pods/c0da055f-fe92-4ff0-94c4-34173ba81de6/volumes" Jul 08 18:29:39.770612 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:39.770576 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8n2h8" Jul 08 18:29:57.287417 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.287336 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78c7f688d6-gpghj"] Jul 08 18:29:57.287796 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.287651 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0da055f-fe92-4ff0-94c4-34173ba81de6" containerName="authorino" Jul 08 18:29:57.287796 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.287666 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0da055f-fe92-4ff0-94c4-34173ba81de6" containerName="authorino" Jul 08 18:29:57.287796 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.287717 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0da055f-fe92-4ff0-94c4-34173ba81de6" containerName="authorino" Jul 08 18:29:57.310820 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.310789 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78c7f688d6-gpghj"] Jul 08 18:29:57.310963 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.310900 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:57.313195 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.313176 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t5ps7\"" Jul 08 18:29:57.351857 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.351830 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78c7f688d6-gpghj"] Jul 08 18:29:57.352070 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:29:57.352052 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fx5x2], unattached volumes=[], failed to process volumes=[kube-api-access-fx5x2]: context canceled" pod="kuadrant-system/authorino-78c7f688d6-gpghj" podUID="4c9ac4c0-bf69-44d7-85d7-7738d9ac9738" Jul 08 18:29:57.398087 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.398058 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5x2\" (UniqueName: \"kubernetes.io/projected/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738-kube-api-access-fx5x2\") pod \"authorino-78c7f688d6-gpghj\" (UID: \"4c9ac4c0-bf69-44d7-85d7-7738d9ac9738\") " pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:57.499368 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.499326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5x2\" (UniqueName: \"kubernetes.io/projected/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738-kube-api-access-fx5x2\") pod \"authorino-78c7f688d6-gpghj\" (UID: \"4c9ac4c0-bf69-44d7-85d7-7738d9ac9738\") " pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:57.507314 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.507288 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5x2\" (UniqueName: \"kubernetes.io/projected/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738-kube-api-access-fx5x2\") pod \"authorino-78c7f688d6-gpghj\" (UID: \"4c9ac4c0-bf69-44d7-85d7-7738d9ac9738\") " pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:57.882755 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.882724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:57.893075 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:57.893049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:58.003624 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:58.003592 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx5x2\" (UniqueName: \"kubernetes.io/projected/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738-kube-api-access-fx5x2\") pod \"4c9ac4c0-bf69-44d7-85d7-7738d9ac9738\" (UID: \"4c9ac4c0-bf69-44d7-85d7-7738d9ac9738\") " Jul 08 18:29:58.005873 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:58.005851 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738-kube-api-access-fx5x2" (OuterVolumeSpecName: "kube-api-access-fx5x2") pod "4c9ac4c0-bf69-44d7-85d7-7738d9ac9738" (UID: "4c9ac4c0-bf69-44d7-85d7-7738d9ac9738"). InnerVolumeSpecName "kube-api-access-fx5x2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:29:58.104649 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:58.104609 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx5x2\" (UniqueName: \"kubernetes.io/projected/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738-kube-api-access-fx5x2\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:29:58.886146 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:58.886114 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78c7f688d6-gpghj" Jul 08 18:29:58.918344 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:58.918313 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-78c7f688d6-gpghj"] Jul 08 18:29:58.921804 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:58.921779 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-78c7f688d6-gpghj"] Jul 08 18:29:59.326707 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:29:59.326677 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c9ac4c0-bf69-44d7-85d7-7738d9ac9738" path="/var/lib/kubelet/pods/4c9ac4c0-bf69-44d7-85d7-7738d9ac9738/volumes" Jul 08 18:30:38.122504 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:30:38.122471 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:30:54.277282 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:30:54.277228 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:31:16.571722 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:31:16.571686 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:31:31.378079 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:31:31.378002 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:31:36.870824 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:31:36.870786 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:32:04.877624 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.877587 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw"] Jul 08 18:32:04.879867 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.879845 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:04.883175 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.883143 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 18:32:04.883324 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.883175 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 18:32:04.883324 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.883198 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 18:32:04.883324 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.883218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rszwk\"" Jul 08 18:32:04.891255 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.891235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw"] Jul 08 18:32:04.980893 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.980854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:04.980893 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.980896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:04.981151 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.980922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:04.981151 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.980971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77qw7\" (UniqueName: \"kubernetes.io/projected/78cfb526-6bcc-4556-8bbb-0385ad868694-kube-api-access-77qw7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:04.981151 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.981057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:04.981151 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:04.981120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78cfb526-6bcc-4556-8bbb-0385ad868694-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082402 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78cfb526-6bcc-4556-8bbb-0385ad868694-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082402 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082676 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082676 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082676 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77qw7\" (UniqueName: \"kubernetes.io/projected/78cfb526-6bcc-4556-8bbb-0385ad868694-kube-api-access-77qw7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082676 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082494 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082934 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082912 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082974 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.082974 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.082967 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.084893 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.084873 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78cfb526-6bcc-4556-8bbb-0385ad868694-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.085041 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.085025 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78cfb526-6bcc-4556-8bbb-0385ad868694-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.091132 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.091109 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77qw7\" (UniqueName: \"kubernetes.io/projected/78cfb526-6bcc-4556-8bbb-0385ad868694-kube-api-access-77qw7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwbgw\" (UID: \"78cfb526-6bcc-4556-8bbb-0385ad868694\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.189170 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.189064 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:05.314946 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.314919 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw"] Jul 08 18:32:05.317306 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:32:05.317277 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cfb526_6bcc_4556_8bbb_0385ad868694.slice/crio-3ca2d0116b8e50dff29f628decc3b2f0151a7681d3333105a2aba3907187ba18 WatchSource:0}: Error finding container 3ca2d0116b8e50dff29f628decc3b2f0151a7681d3333105a2aba3907187ba18: Status 404 returned error can't find the container with id 3ca2d0116b8e50dff29f628decc3b2f0151a7681d3333105a2aba3907187ba18 Jul 08 18:32:05.367184 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:05.367147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" event={"ID":"78cfb526-6bcc-4556-8bbb-0385ad868694","Type":"ContainerStarted","Data":"3ca2d0116b8e50dff29f628decc3b2f0151a7681d3333105a2aba3907187ba18"} Jul 08 18:32:06.666188 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:06.666148 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:32:07.377000 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:07.376963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" event={"ID":"78cfb526-6bcc-4556-8bbb-0385ad868694","Type":"ContainerStarted","Data":"d8a9e66ab5bbfeec925c0284947a9cf3e0b355bee17c2a69922b12b5bd506444"} Jul 08 18:32:07.394939 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:07.394891 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" podStartSLOduration=2.248984829 podStartE2EDuration="3.394842632s" totalImagesPullingTime="1.145857803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:32:07.393807715 +0000 UTC m=+840.655873656" watchObservedRunningTime="2026-07-08 18:32:07.394842632 +0000 UTC m=+840.656908386" Jul 08 18:32:15.189489 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:15.189450 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:15.189974 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:15.189537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:15.201801 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:15.201775 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:15.420324 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:15.420298 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwbgw" Jul 08 18:32:18.962161 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.962124 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn"] Jul 08 18:32:18.968281 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.965337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:18.968642 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.968619 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 18:32:18.977390 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.977363 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn"] Jul 08 18:32:18.999321 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.999292 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:18.999490 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.999330 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pfdr\" (UniqueName: \"kubernetes.io/projected/6453e115-4ad0-4197-82ad-bee4e2f931ee-kube-api-access-6pfdr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:18.999490 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.999393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:18.999490 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.999431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6453e115-4ad0-4197-82ad-bee4e2f931ee-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:18.999627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.999502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:18.999627 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:18.999540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100103 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100103 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6453e115-4ad0-4197-82ad-bee4e2f931ee-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100103 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100123 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100461 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100461 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100461 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pfdr\" (UniqueName: \"kubernetes.io/projected/6453e115-4ad0-4197-82ad-bee4e2f931ee-kube-api-access-6pfdr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100621 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100667 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100622 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.100667 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.100652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.102594 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.102565 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6453e115-4ad0-4197-82ad-bee4e2f931ee-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.102834 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.102813 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6453e115-4ad0-4197-82ad-bee4e2f931ee-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.108067 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.108042 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pfdr\" (UniqueName: \"kubernetes.io/projected/6453e115-4ad0-4197-82ad-bee4e2f931ee-kube-api-access-6pfdr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn\" (UID: \"6453e115-4ad0-4197-82ad-bee4e2f931ee\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.280104 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.280065 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:19.408283 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.408241 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn"] Jul 08 18:32:19.410982 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:32:19.410950 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6453e115_4ad0_4197_82ad_bee4e2f931ee.slice/crio-6eff98043a86f57cedd7642d761aca50db5e5819d769a6b2271fe1abfbc85a50 WatchSource:0}: Error finding container 6eff98043a86f57cedd7642d761aca50db5e5819d769a6b2271fe1abfbc85a50: Status 404 returned error can't find the container with id 6eff98043a86f57cedd7642d761aca50db5e5819d769a6b2271fe1abfbc85a50 Jul 08 18:32:19.425238 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:19.425203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" event={"ID":"6453e115-4ad0-4197-82ad-bee4e2f931ee","Type":"ContainerStarted","Data":"6eff98043a86f57cedd7642d761aca50db5e5819d769a6b2271fe1abfbc85a50"} Jul 08 18:32:20.431829 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:20.431792 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" event={"ID":"6453e115-4ad0-4197-82ad-bee4e2f931ee","Type":"ContainerStarted","Data":"3ded6ede2677bb87f23c5f37603f740c1df9b99ced0f3984407405a69b083ce2"} Jul 08 18:32:20.449732 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:20.449685 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" podStartSLOduration=2.261857025 podStartE2EDuration="2.449669486s" totalImagesPullingTime="187.812461ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:32:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:32:20.448100718 +0000 UTC m=+853.710166471" watchObservedRunningTime="2026-07-08 18:32:20.449669486 +0000 UTC m=+853.711735286" Jul 08 18:32:21.371586 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:21.371548 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:32:29.280833 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:29.280776 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:29.281366 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:29.280867 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:29.293349 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:29.293327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:32:29.479999 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:32:29.479968 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn" Jul 08 18:33:09.551654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.551618 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dff76d979-cqcl7"] Jul 08 18:33:09.554102 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.554085 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.557473 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.557450 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 18:33:09.557595 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.557450 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t5ps7\"" Jul 08 18:33:09.562057 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.562037 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dff76d979-cqcl7"] Jul 08 18:33:09.624052 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.624016 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rq9r\" (UniqueName: \"kubernetes.io/projected/daa8ee36-4b69-44ca-a40f-cfbe515799ee-kube-api-access-7rq9r\") pod \"authorino-5dff76d979-cqcl7\" (UID: \"daa8ee36-4b69-44ca-a40f-cfbe515799ee\") " pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.624226 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.624078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daa8ee36-4b69-44ca-a40f-cfbe515799ee-tls-cert\") pod \"authorino-5dff76d979-cqcl7\" (UID: \"daa8ee36-4b69-44ca-a40f-cfbe515799ee\") " pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.725165 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.725125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daa8ee36-4b69-44ca-a40f-cfbe515799ee-tls-cert\") pod \"authorino-5dff76d979-cqcl7\" (UID: \"daa8ee36-4b69-44ca-a40f-cfbe515799ee\") " pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.725371 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.725184 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rq9r\" (UniqueName: \"kubernetes.io/projected/daa8ee36-4b69-44ca-a40f-cfbe515799ee-kube-api-access-7rq9r\") pod \"authorino-5dff76d979-cqcl7\" (UID: \"daa8ee36-4b69-44ca-a40f-cfbe515799ee\") " pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.727802 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.727781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daa8ee36-4b69-44ca-a40f-cfbe515799ee-tls-cert\") pod \"authorino-5dff76d979-cqcl7\" (UID: \"daa8ee36-4b69-44ca-a40f-cfbe515799ee\") " pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.733054 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.733030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rq9r\" (UniqueName: \"kubernetes.io/projected/daa8ee36-4b69-44ca-a40f-cfbe515799ee-kube-api-access-7rq9r\") pod \"authorino-5dff76d979-cqcl7\" (UID: \"daa8ee36-4b69-44ca-a40f-cfbe515799ee\") " pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.863604 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.863521 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dff76d979-cqcl7" Jul 08 18:33:09.995689 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:09.995664 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dff76d979-cqcl7"] Jul 08 18:33:09.997992 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:33:09.997953 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaa8ee36_4b69_44ca_a40f_cfbe515799ee.slice/crio-2bd285e6a0dd59839af4201672b69c49cc6a4e2fefbdb576e78decb737cacedf WatchSource:0}: Error finding container 2bd285e6a0dd59839af4201672b69c49cc6a4e2fefbdb576e78decb737cacedf: Status 404 returned error can't find the container with id 2bd285e6a0dd59839af4201672b69c49cc6a4e2fefbdb576e78decb737cacedf Jul 08 18:33:10.640631 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:10.640540 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dff76d979-cqcl7" event={"ID":"daa8ee36-4b69-44ca-a40f-cfbe515799ee","Type":"ContainerStarted","Data":"b9743df8d4e72731136023556a22b7197c872f6c9bbecc62729bc8ffb5b3b949"} Jul 08 18:33:10.640631 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:10.640585 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dff76d979-cqcl7" event={"ID":"daa8ee36-4b69-44ca-a40f-cfbe515799ee","Type":"ContainerStarted","Data":"2bd285e6a0dd59839af4201672b69c49cc6a4e2fefbdb576e78decb737cacedf"} Jul 08 18:33:10.658452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:10.658389 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5dff76d979-cqcl7" podStartSLOduration=1.310684878 podStartE2EDuration="1.658366161s" totalImagesPullingTime="347.681283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:33:10.656136014 +0000 UTC m=+903.918201765" watchObservedRunningTime="2026-07-08 18:33:10.658366161 +0000 UTC m=+903.920431914" Jul 08 18:33:38.487315 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:38.484712 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:33:49.274473 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:49.274435 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:33:59.779005 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:33:59.778968 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:34:09.971113 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:34:09.971074 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:34:19.575115 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:34:19.575080 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:34:48.772504 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:34:48.772420 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:34:59.868457 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:34:59.868424 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:35:33.371319 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:35:33.371288 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:35:42.777928 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:35:42.777885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:35:50.466345 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:35:50.466299 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:35:58.065914 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:35:58.065833 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:36:08.869495 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:36:08.869459 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:36:20.680886 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:36:20.680845 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:36:31.976119 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:36:31.976087 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:36:42.067062 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:36:42.067032 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:37:03.977514 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:37:03.977475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:37:13.170771 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:37:13.170737 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:37:35.774028 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:37:35.773930 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:37:47.069214 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:37:47.069182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:38:04.677478 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:38:04.677436 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:38:14.679003 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:38:14.677918 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:38:24.468765 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:38:24.468727 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:38:33.962919 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:38:33.962885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:38:42.366236 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:38:42.366200 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:40:31.566206 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:40:31.566113 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:40:42.901175 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:40:42.901135 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:40:51.578218 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:40:51.578179 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:41:03.170362 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:41:03.170323 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:41:14.671252 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:41:14.671217 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:41:25.473007 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:41:25.472970 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:41:33.867279 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:41:33.867240 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:41:44.771472 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:41:44.771440 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:41:56.062577 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:41:56.062541 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:42:05.164712 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:42:05.164627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:42:15.468964 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:42:15.468926 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:42:37.876407 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:42:37.876373 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:42:47.769422 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:42:47.769374 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:42:57.666298 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:42:57.666244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:43:07.072748 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:43:07.072711 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:43:12.470734 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:43:12.470697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:43:28.576407 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:43:28.576315 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:43:39.071756 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:43:39.071719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:43:48.471757 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:43:48.471722 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:43:58.374974 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:43:58.374926 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:44:07.064521 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:44:07.064484 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:44:18.565177 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:44:18.565142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:44:31.173169 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:44:31.173129 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:44:40.670019 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:44:40.669982 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:44:49.576016 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:44:49.575977 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:45:00.965991 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:45:00.965912 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:45:12.379144 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:45:12.379106 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:45:21.470908 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:45:21.470861 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:45:33.366215 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:45:33.366182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:45:45.775707 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:45:45.775670 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:45:54.663203 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:45:54.663166 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:46:03.163225 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:46:03.163183 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:46:12.074899 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:46:12.074860 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:46:33.270594 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:46:33.270504 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:46:43.679446 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:46:43.679408 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:46:53.789836 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:46:53.789793 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:47:14.391854 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:47:14.391816 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:47:22.877332 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:47:22.877294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:48:44.263982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:48:44.263932 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:48:52.766739 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:48:52.766703 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:49:04.469858 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:04.469823 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:49:16.267470 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:16.267434 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:49:48.868654 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:48.868572 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:49:53.474732 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.474695 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr"] Jul 08 18:49:53.477633 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.477615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.480200 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.480179 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f330a831\"/\"kube-root-ca.crt\"" Jul 08 18:49:53.481049 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.481031 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f330a831\"/\"default-dockercfg-w78c6\"" Jul 08 18:49:53.481159 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.481078 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f330a831\"/\"openshift-service-ca.crt\"" Jul 08 18:49:53.481220 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.481211 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f330a831\"/\"e2e-lifecycle-model-f330a831-kserve-self-signed-certs\"" Jul 08 18:49:53.490785 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.490765 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr"] Jul 08 18:49:53.577610 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.577573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-dshm\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.577776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.577639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snhhm\" (UniqueName: \"kubernetes.io/projected/b54b67f3-e607-4756-830f-3e9e8289e3dd-kube-api-access-snhhm\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.577776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.577664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-tmp-dir\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.577776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.577683 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-model-cache\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.577776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.577702 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-home\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.577776 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.577721 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54b67f3-e607-4756-830f-3e9e8289e3dd-tls-certs\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678580 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-model-cache\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678713 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-home\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678713 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54b67f3-e607-4756-830f-3e9e8289e3dd-tls-certs\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678713 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-dshm\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678863 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snhhm\" (UniqueName: \"kubernetes.io/projected/b54b67f3-e607-4756-830f-3e9e8289e3dd-kube-api-access-snhhm\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678863 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-tmp-dir\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.678965 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.678942 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-model-cache\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.679016 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.679001 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-home\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.679090 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.679067 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-tmp-dir\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.681179 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.681155 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-dshm\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.681362 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.681343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54b67f3-e607-4756-830f-3e9e8289e3dd-tls-certs\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.687827 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.687782 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snhhm\" (UniqueName: \"kubernetes.io/projected/b54b67f3-e607-4756-830f-3e9e8289e3dd-kube-api-access-snhhm\") pod \"e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.786673 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.786573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:49:53.916519 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:53.916487 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr"] Jul 08 18:49:53.919878 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:49:53.919846 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb54b67f3_e607_4756_830f_3e9e8289e3dd.slice/crio-f60bf52dcfae789d18e272d8f709559f9ca9ec6f82f9b8da2cc6c0936244fffa WatchSource:0}: Error finding container f60bf52dcfae789d18e272d8f709559f9ca9ec6f82f9b8da2cc6c0936244fffa: Status 404 returned error can't find the container with id f60bf52dcfae789d18e272d8f709559f9ca9ec6f82f9b8da2cc6c0936244fffa Jul 08 18:49:54.788997 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:54.788955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" event={"ID":"b54b67f3-e607-4756-830f-3e9e8289e3dd","Type":"ContainerStarted","Data":"66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd"} Jul 08 18:49:54.789469 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:54.789005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" event={"ID":"b54b67f3-e607-4756-830f-3e9e8289e3dd","Type":"ContainerStarted","Data":"f60bf52dcfae789d18e272d8f709559f9ca9ec6f82f9b8da2cc6c0936244fffa"} Jul 08 18:49:54.807109 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:54.807050 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" podStartSLOduration=1.8070354709999998 podStartE2EDuration="1.807035471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:49:54.805876617 +0000 UTC m=+1908.067942370" watchObservedRunningTime="2026-07-08 18:49:54.807035471 +0000 UTC m=+1908.069101222" Jul 08 18:49:57.670347 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:49:57.670308 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:50:03.787224 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:03.787165 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:50:03.787224 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:03.787236 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:50:03.799681 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:03.799657 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:50:03.838120 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:03.838094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:50:21.472083 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:21.472047 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:50:34.564566 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:34.564526 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:50:35.026324 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:35.026284 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr"] Jul 08 18:50:50.047359 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.047322 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" podUID="b54b67f3-e607-4756-830f-3e9e8289e3dd" containerName="main" containerID="cri-o://66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd" gracePeriod=45 Jul 08 18:50:50.295300 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.295257 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:50:50.375337 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375228 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-tmp-dir\") pod \"b54b67f3-e607-4756-830f-3e9e8289e3dd\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " Jul 08 18:50:50.375337 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375287 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54b67f3-e607-4756-830f-3e9e8289e3dd-tls-certs\") pod \"b54b67f3-e607-4756-830f-3e9e8289e3dd\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " Jul 08 18:50:50.375337 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375319 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-home\") pod \"b54b67f3-e607-4756-830f-3e9e8289e3dd\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " Jul 08 18:50:50.375618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375347 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-dshm\") pod \"b54b67f3-e607-4756-830f-3e9e8289e3dd\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " Jul 08 18:50:50.375618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375383 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snhhm\" (UniqueName: \"kubernetes.io/projected/b54b67f3-e607-4756-830f-3e9e8289e3dd-kube-api-access-snhhm\") pod \"b54b67f3-e607-4756-830f-3e9e8289e3dd\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " Jul 08 18:50:50.375618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375413 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-model-cache\") pod \"b54b67f3-e607-4756-830f-3e9e8289e3dd\" (UID: \"b54b67f3-e607-4756-830f-3e9e8289e3dd\") " Jul 08 18:50:50.375618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375479 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b54b67f3-e607-4756-830f-3e9e8289e3dd" (UID: "b54b67f3-e607-4756-830f-3e9e8289e3dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:50:50.375793 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375639 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-home" (OuterVolumeSpecName: "home") pod "b54b67f3-e607-4756-830f-3e9e8289e3dd" (UID: "b54b67f3-e607-4756-830f-3e9e8289e3dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:50:50.375793 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375678 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-tmp-dir\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:50:50.375793 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.375739 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "b54b67f3-e607-4756-830f-3e9e8289e3dd" (UID: "b54b67f3-e607-4756-830f-3e9e8289e3dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:50:50.377855 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.377824 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-dshm" (OuterVolumeSpecName: "dshm") pod "b54b67f3-e607-4756-830f-3e9e8289e3dd" (UID: "b54b67f3-e607-4756-830f-3e9e8289e3dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:50:50.377956 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.377874 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b54b67f3-e607-4756-830f-3e9e8289e3dd-kube-api-access-snhhm" (OuterVolumeSpecName: "kube-api-access-snhhm") pod "b54b67f3-e607-4756-830f-3e9e8289e3dd" (UID: "b54b67f3-e607-4756-830f-3e9e8289e3dd"). InnerVolumeSpecName "kube-api-access-snhhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:50:50.377956 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.377891 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b54b67f3-e607-4756-830f-3e9e8289e3dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b54b67f3-e607-4756-830f-3e9e8289e3dd" (UID: "b54b67f3-e607-4756-830f-3e9e8289e3dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:50:50.476468 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.476422 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-home\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:50:50.476468 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.476463 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-dshm\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:50:50.476468 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.476473 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snhhm\" (UniqueName: \"kubernetes.io/projected/b54b67f3-e607-4756-830f-3e9e8289e3dd-kube-api-access-snhhm\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:50:50.476468 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.476486 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b54b67f3-e607-4756-830f-3e9e8289e3dd-model-cache\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:50:50.476734 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:50.476504 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b54b67f3-e607-4756-830f-3e9e8289e3dd-tls-certs\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:50:51.038041 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.038005 2562 generic.go:356] "Generic (PLEG): container finished" podID="b54b67f3-e607-4756-830f-3e9e8289e3dd" containerID="66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd" exitCode=0 Jul 08 18:50:51.038213 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.038049 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" Jul 08 18:50:51.038213 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.038084 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" event={"ID":"b54b67f3-e607-4756-830f-3e9e8289e3dd","Type":"ContainerDied","Data":"66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd"} Jul 08 18:50:51.038213 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.038118 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr" event={"ID":"b54b67f3-e607-4756-830f-3e9e8289e3dd","Type":"ContainerDied","Data":"f60bf52dcfae789d18e272d8f709559f9ca9ec6f82f9b8da2cc6c0936244fffa"} Jul 08 18:50:51.038213 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.038137 2562 scope.go:117] "RemoveContainer" containerID="66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd" Jul 08 18:50:51.048888 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.048670 2562 scope.go:117] "RemoveContainer" containerID="66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd" Jul 08 18:50:51.049155 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:50:51.049088 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd\": container with ID starting with 66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd not found: ID does not exist" containerID="66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd" Jul 08 18:50:51.049155 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.049114 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd"} err="failed to get container status \"66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd\": rpc error: code = NotFound desc = could not find container \"66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd\": container with ID starting with 66fa0a841b1a7e2541251cc2d6f04ae76a9beec6a0ae196c68828ad434071ffd not found: ID does not exist" Jul 08 18:50:51.058293 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.058247 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr"] Jul 08 18:50:51.061968 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.061944 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f330a831/e2e-lifecycle-model-f330a831-kserve-6b9485fc95-dx4zr"] Jul 08 18:50:51.326960 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:50:51.326881 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b54b67f3-e607-4756-830f-3e9e8289e3dd" path="/var/lib/kubelet/pods/b54b67f3-e607-4756-830f-3e9e8289e3dd/volumes" Jul 08 18:51:14.566664 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:51:14.566629 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:51:35.981232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:51:35.981187 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:51:49.869919 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:51:49.869878 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:52:38.369689 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:52:38.369603 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:52:49.778710 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:52:49.778675 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:53:10.970071 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.970038 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf"] Jul 08 18:53:10.970500 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.970329 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b54b67f3-e607-4756-830f-3e9e8289e3dd" containerName="main" Jul 08 18:53:10.970500 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.970341 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b54b67f3-e607-4756-830f-3e9e8289e3dd" containerName="main" Jul 08 18:53:10.970500 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.970421 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b54b67f3-e607-4756-830f-3e9e8289e3dd" containerName="main" Jul 08 18:53:10.973071 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.973053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:10.976190 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.976167 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-200f0c\"/\"default-dockercfg-2xqtm\"" Jul 08 18:53:10.976341 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.976196 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-200f0c\"/\"test-model-200f0c-kserve-self-signed-certs\"" Jul 08 18:53:10.976341 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.976243 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-200f0c\"/\"openshift-service-ca.crt\"" Jul 08 18:53:10.976341 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.976250 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-200f0c\"/\"kube-root-ca.crt\"" Jul 08 18:53:10.982082 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:10.982062 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf"] Jul 08 18:53:11.055524 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.055487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tls-certs\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.055705 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.055536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-model-cache\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.055705 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.055562 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tmp-dir\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.055705 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.055609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-home\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.055705 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.055649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-dshm\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.055705 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.055677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sft5n\" (UniqueName: \"kubernetes.io/projected/22d2f32a-33a3-45a2-b9d0-0862256e1c42-kube-api-access-sft5n\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156160 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sft5n\" (UniqueName: \"kubernetes.io/projected/22d2f32a-33a3-45a2-b9d0-0862256e1c42-kube-api-access-sft5n\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156393 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156190 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tls-certs\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156393 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-model-cache\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156393 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tmp-dir\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156565 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156419 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-home\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156565 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-dshm\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156699 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156681 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-model-cache\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156751 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tmp-dir\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.156751 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.156726 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-home\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.158816 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.158796 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-dshm\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.159106 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.159084 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tls-certs\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.164429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.164412 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sft5n\" (UniqueName: \"kubernetes.io/projected/22d2f32a-33a3-45a2-b9d0-0862256e1c42-kube-api-access-sft5n\") pod \"test-model-200f0c-kserve-5bdf6c9f5f-qmxnf\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.282410 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.282366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:11.413758 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.413725 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf"] Jul 08 18:53:11.416820 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:53:11.416793 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22d2f32a_33a3_45a2_b9d0_0862256e1c42.slice/crio-3a7be1ff03a9d30ddadf7db1c711d8e3c646a225c510391338fb1141282b4450 WatchSource:0}: Error finding container 3a7be1ff03a9d30ddadf7db1c711d8e3c646a225c510391338fb1141282b4450: Status 404 returned error can't find the container with id 3a7be1ff03a9d30ddadf7db1c711d8e3c646a225c510391338fb1141282b4450 Jul 08 18:53:11.619340 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.619220 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" event={"ID":"22d2f32a-33a3-45a2-b9d0-0862256e1c42","Type":"ContainerStarted","Data":"fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83"} Jul 08 18:53:11.619340 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.619292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" event={"ID":"22d2f32a-33a3-45a2-b9d0-0862256e1c42","Type":"ContainerStarted","Data":"3a7be1ff03a9d30ddadf7db1c711d8e3c646a225c510391338fb1141282b4450"} Jul 08 18:53:11.638017 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:11.637966 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" podStartSLOduration=1.637951205 podStartE2EDuration="1.637951205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:53:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:53:11.636903095 +0000 UTC m=+2104.898968847" watchObservedRunningTime="2026-07-08 18:53:11.637951205 +0000 UTC m=+2104.900016936" Jul 08 18:53:14.968843 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:14.968805 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:53:21.283592 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:21.283537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:21.284097 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:21.283620 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:21.296052 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:21.296025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:21.671343 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:21.671239 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:53:25.666517 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:25.666483 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:53:33.872322 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.872290 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7"] Jul 08 18:53:33.875901 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.875886 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:33.878345 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.878254 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-095308\"/\"openshift-service-ca.crt\"" Jul 08 18:53:33.879170 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.879155 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-095308\"/\"test-model-095308-kserve-self-signed-certs\"" Jul 08 18:53:33.879248 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.879222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-095308\"/\"kube-root-ca.crt\"" Jul 08 18:53:33.879343 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.879299 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-095308\"/\"default-dockercfg-dscjq\"" Jul 08 18:53:33.885583 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.885564 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7"] Jul 08 18:53:33.951308 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.951241 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-home\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:33.951486 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.951321 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc2j5\" (UniqueName: \"kubernetes.io/projected/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-kube-api-access-kc2j5\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:33.951486 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.951346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tls-certs\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:33.951486 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.951376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-dshm\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:33.951486 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.951464 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-model-cache\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:33.951614 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:33.951493 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tmp-dir\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052232 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-dshm\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052390 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-model-cache\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052390 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tmp-dir\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052390 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-home\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052390 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc2j5\" (UniqueName: \"kubernetes.io/projected/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-kube-api-access-kc2j5\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052551 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tls-certs\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052665 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052644 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-model-cache\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052730 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052693 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tmp-dir\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.052815 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.052789 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-home\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.054703 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.054685 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-dshm\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.055025 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.055002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tls-certs\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.060344 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.060312 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc2j5\" (UniqueName: \"kubernetes.io/projected/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-kube-api-access-kc2j5\") pod \"test-model-095308-kserve-5db6dcf755-kkgp7\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.185618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.185522 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:34.317241 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.317059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7"] Jul 08 18:53:34.319971 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:53:34.319943 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeadcef4c_f1be_4d8c_99a3_1e3bc64ca2c1.slice/crio-5acceed41cd59df08f752b6ca488f10dacfbea7fb64060cbeca8d8355741d6d2 WatchSource:0}: Error finding container 5acceed41cd59df08f752b6ca488f10dacfbea7fb64060cbeca8d8355741d6d2: Status 404 returned error can't find the container with id 5acceed41cd59df08f752b6ca488f10dacfbea7fb64060cbeca8d8355741d6d2 Jul 08 18:53:34.713439 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.713394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" event={"ID":"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1","Type":"ContainerStarted","Data":"fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f"} Jul 08 18:53:34.713439 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.713441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" event={"ID":"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1","Type":"ContainerStarted","Data":"5acceed41cd59df08f752b6ca488f10dacfbea7fb64060cbeca8d8355741d6d2"} Jul 08 18:53:34.729595 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:34.729535 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" podStartSLOduration=1.729519107 podStartE2EDuration="1.729519107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:53:34.728907024 +0000 UTC m=+2127.990972777" watchObservedRunningTime="2026-07-08 18:53:34.729519107 +0000 UTC m=+2127.991584858" Jul 08 18:53:40.368621 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:40.368584 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:53:44.186085 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:44.186043 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:44.186085 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:44.186093 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:44.197993 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:44.197970 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:44.770309 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:44.770276 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:53:56.039456 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:56.039422 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf"] Jul 08 18:53:56.958933 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:53:56.958856 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7"] Jul 08 18:54:02.372809 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:02.372770 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:54:11.059194 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.059127 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" podUID="22d2f32a-33a3-45a2-b9d0-0862256e1c42" containerName="main" containerID="cri-o://fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83" gracePeriod=45 Jul 08 18:54:11.301786 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.301752 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:54:11.456258 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456154 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-home\") pod \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " Jul 08 18:54:11.456258 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sft5n\" (UniqueName: \"kubernetes.io/projected/22d2f32a-33a3-45a2-b9d0-0862256e1c42-kube-api-access-sft5n\") pod \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " Jul 08 18:54:11.456258 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456225 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-dshm\") pod \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " Jul 08 18:54:11.456559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456305 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tls-certs\") pod \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " Jul 08 18:54:11.456559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456336 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-model-cache\") pod \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " Jul 08 18:54:11.456559 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456358 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tmp-dir\") pod \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\" (UID: \"22d2f32a-33a3-45a2-b9d0-0862256e1c42\") " Jul 08 18:54:11.456695 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456549 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-home" (OuterVolumeSpecName: "home") pod "22d2f32a-33a3-45a2-b9d0-0862256e1c42" (UID: "22d2f32a-33a3-45a2-b9d0-0862256e1c42"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:11.456695 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456638 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-model-cache" (OuterVolumeSpecName: "model-cache") pod "22d2f32a-33a3-45a2-b9d0-0862256e1c42" (UID: "22d2f32a-33a3-45a2-b9d0-0862256e1c42"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:11.456695 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.456685 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22d2f32a-33a3-45a2-b9d0-0862256e1c42" (UID: "22d2f32a-33a3-45a2-b9d0-0862256e1c42"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:11.458618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.458590 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22d2f32a-33a3-45a2-b9d0-0862256e1c42" (UID: "22d2f32a-33a3-45a2-b9d0-0862256e1c42"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:54:11.458618 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.458593 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-dshm" (OuterVolumeSpecName: "dshm") pod "22d2f32a-33a3-45a2-b9d0-0862256e1c42" (UID: "22d2f32a-33a3-45a2-b9d0-0862256e1c42"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:11.458754 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.458660 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d2f32a-33a3-45a2-b9d0-0862256e1c42-kube-api-access-sft5n" (OuterVolumeSpecName: "kube-api-access-sft5n") pod "22d2f32a-33a3-45a2-b9d0-0862256e1c42" (UID: "22d2f32a-33a3-45a2-b9d0-0862256e1c42"). InnerVolumeSpecName "kube-api-access-sft5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:11.557134 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.557093 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-home\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:11.557134 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.557126 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sft5n\" (UniqueName: \"kubernetes.io/projected/22d2f32a-33a3-45a2-b9d0-0862256e1c42-kube-api-access-sft5n\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:11.557134 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.557141 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-dshm\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:11.557405 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.557151 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tls-certs\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:11.557405 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.557163 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-model-cache\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:11.557405 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.557172 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22d2f32a-33a3-45a2-b9d0-0862256e1c42-tmp-dir\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:11.884724 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.884694 2562 generic.go:356] "Generic (PLEG): container finished" podID="22d2f32a-33a3-45a2-b9d0-0862256e1c42" containerID="fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83" exitCode=0 Jul 08 18:54:11.884938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.884741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" event={"ID":"22d2f32a-33a3-45a2-b9d0-0862256e1c42","Type":"ContainerDied","Data":"fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83"} Jul 08 18:54:11.884938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.884747 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" Jul 08 18:54:11.884938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.884776 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf" event={"ID":"22d2f32a-33a3-45a2-b9d0-0862256e1c42","Type":"ContainerDied","Data":"3a7be1ff03a9d30ddadf7db1c711d8e3c646a225c510391338fb1141282b4450"} Jul 08 18:54:11.884938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.884793 2562 scope.go:117] "RemoveContainer" containerID="fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83" Jul 08 18:54:11.893072 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.893055 2562 scope.go:117] "RemoveContainer" containerID="fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83" Jul 08 18:54:11.893377 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:54:11.893358 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83\": container with ID starting with fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83 not found: ID does not exist" containerID="fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83" Jul 08 18:54:11.893426 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.893387 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83"} err="failed to get container status \"fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83\": rpc error: code = NotFound desc = could not find container \"fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83\": container with ID starting with fd3ef27150bd2d6e79e4bd4a0f44641e6c3e9c5adaced393bb0884592b5e0a83 not found: ID does not exist" Jul 08 18:54:11.904517 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.904492 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf"] Jul 08 18:54:11.908405 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.908384 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-200f0c/test-model-200f0c-kserve-5bdf6c9f5f-qmxnf"] Jul 08 18:54:11.978817 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:11.978772 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" podUID="eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" containerName="main" containerID="cri-o://fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f" gracePeriod=45 Jul 08 18:54:12.227102 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.227078 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:54:12.262035 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262006 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc2j5\" (UniqueName: \"kubernetes.io/projected/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-kube-api-access-kc2j5\") pod \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " Jul 08 18:54:12.262229 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262044 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-dshm\") pod \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " Jul 08 18:54:12.262229 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262069 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tls-certs\") pod \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " Jul 08 18:54:12.262229 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262139 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-home\") pod \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " Jul 08 18:54:12.262229 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262178 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tmp-dir\") pod \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " Jul 08 18:54:12.262229 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262204 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-model-cache\") pod \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\" (UID: \"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1\") " Jul 08 18:54:12.262507 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262468 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-home" (OuterVolumeSpecName: "home") pod "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" (UID: "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:12.262507 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262484 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" (UID: "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:12.262624 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.262604 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" (UID: "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:12.264598 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.264574 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-dshm" (OuterVolumeSpecName: "dshm") pod "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" (UID: "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 18:54:12.264938 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.264920 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" (UID: "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 18:54:12.265006 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.264933 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-kube-api-access-kc2j5" (OuterVolumeSpecName: "kube-api-access-kc2j5") pod "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" (UID: "eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1"). InnerVolumeSpecName "kube-api-access-kc2j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 18:54:12.362860 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.362815 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tmp-dir\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:12.362860 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.362854 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-model-cache\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:12.362860 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.362867 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc2j5\" (UniqueName: \"kubernetes.io/projected/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-kube-api-access-kc2j5\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:12.363100 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.362880 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-dshm\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:12.363100 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.362892 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-tls-certs\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:12.363100 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.362902 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1-home\") on node \"ip-10-0-131-106.ec2.internal\" DevicePath \"\"" Jul 08 18:54:12.890594 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.890558 2562 generic.go:356] "Generic (PLEG): container finished" podID="eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" containerID="fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f" exitCode=0 Jul 08 18:54:12.890779 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.890601 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" Jul 08 18:54:12.890779 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.890641 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" event={"ID":"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1","Type":"ContainerDied","Data":"fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f"} Jul 08 18:54:12.890779 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.890686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7" event={"ID":"eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1","Type":"ContainerDied","Data":"5acceed41cd59df08f752b6ca488f10dacfbea7fb64060cbeca8d8355741d6d2"} Jul 08 18:54:12.890779 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.890703 2562 scope.go:117] "RemoveContainer" containerID="fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f" Jul 08 18:54:12.898835 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.898820 2562 scope.go:117] "RemoveContainer" containerID="fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f" Jul 08 18:54:12.899116 ip-10-0-131-106 kubenswrapper[2562]: E0708 18:54:12.899096 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f\": container with ID starting with fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f not found: ID does not exist" containerID="fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f" Jul 08 18:54:12.899165 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.899125 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f"} err="failed to get container status \"fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f\": rpc error: code = NotFound desc = could not find container \"fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f\": container with ID starting with fd94d98d72082bbb5323d2e60f898ceb802fc6fc0f0251f6a9c7e530daa92a5f not found: ID does not exist" Jul 08 18:54:12.910989 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.910965 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7"] Jul 08 18:54:12.917428 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:12.917409 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-095308/test-model-095308-kserve-5db6dcf755-kkgp7"] Jul 08 18:54:13.327186 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:13.327152 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22d2f32a-33a3-45a2-b9d0-0862256e1c42" path="/var/lib/kubelet/pods/22d2f32a-33a3-45a2-b9d0-0862256e1c42/volumes" Jul 08 18:54:13.327570 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:13.327536 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" path="/var/lib/kubelet/pods/eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1/volumes" Jul 08 18:54:43.877161 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:43.877120 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:54:58.387513 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:54:58.387479 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:55:15.275156 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:55:15.275118 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:55:30.478185 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:55:30.478101 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:55:44.469201 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:55:44.469167 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:58:51.070393 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:58:51.070306 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:59:01.611404 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:01.611375 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5dff76d979-cqcl7_daa8ee36-4b69-44ca-a40f-cfbe515799ee/authorino/0.log" Jul 08 18:59:02.970475 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:02.970437 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8n2h8"] Jul 08 18:59:07.293822 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:07.293792 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-zjf4d_87cde5bd-973f-444b-974c-42ddacccd5e9/copy-manifests/0.log" Jul 08 18:59:07.300853 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:07.300824 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-zjf4d_87cde5bd-973f-444b-974c-42ddacccd5e9/manager/0.log" Jul 08 18:59:08.662722 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:08.662668 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5dff76d979-cqcl7_daa8ee36-4b69-44ca-a40f-cfbe515799ee/authorino/0.log" Jul 08 18:59:09.037041 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:09.037013 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5xlpb_645754bc-1a1b-4dce-843a-1634b539e041/kuadrant-console-plugin/0.log" Jul 08 18:59:09.437503 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:09.437418 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8n2h8_89180bfa-5115-4a51-bfb7-056d5e5292b1/limitador/0.log" Jul 08 18:59:10.180594 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:10.180564 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb94956b8-l5brt_ce0dcf26-a30f-4c4c-99b0-bd5693a87118/kube-auth-proxy/0.log" Jul 08 18:59:11.602646 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:11.602612 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vwbgw_78cfb526-6bcc-4556-8bbb-0385ad868694/main/0.log" Jul 08 18:59:11.723483 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:11.723454 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7dwkn_6453e115-4ad0-4197-82ad-bee4e2f931ee/main/0.log" Jul 08 18:59:18.748276 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:18.748242 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j2f2w_b59bbc9f-1106-47b1-ae05-4ebc641b1249/global-pull-secret-syncer/0.log" Jul 08 18:59:18.932187 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:18.932152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wmnmp_acec3e89-3c81-41eb-a93b-72c59f4eb441/konnectivity-agent/0.log" Jul 08 18:59:18.952145 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:18.952118 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-106.ec2.internal_188ef7b029f22b3628e6f2183332d139/haproxy/0.log" Jul 08 18:59:23.938084 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:23.938045 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5dff76d979-cqcl7_daa8ee36-4b69-44ca-a40f-cfbe515799ee/authorino/0.log" Jul 08 18:59:24.067759 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:24.067733 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5xlpb_645754bc-1a1b-4dce-843a-1634b539e041/kuadrant-console-plugin/0.log" Jul 08 18:59:24.310452 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:24.310420 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8n2h8_89180bfa-5115-4a51-bfb7-056d5e5292b1/limitador/0.log" Jul 08 18:59:26.406650 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:26.406624 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5687887474-m2vgb_cb671a63-0697-485a-a177-01c3475259d2/metrics-server/0.log" Jul 08 18:59:26.628369 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:26.628337 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddq2n_05245de8-9d7d-41f8-b9ea-a922c5c8bf7e/node-exporter/0.log" Jul 08 18:59:26.648832 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:26.648809 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddq2n_05245de8-9d7d-41f8-b9ea-a922c5c8bf7e/kube-rbac-proxy/0.log" Jul 08 18:59:26.672918 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:26.672847 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ddq2n_05245de8-9d7d-41f8-b9ea-a922c5c8bf7e/init-textfile/0.log" Jul 08 18:59:28.686922 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.686887 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql"] Jul 08 18:59:28.687382 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.687157 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d2f32a-33a3-45a2-b9d0-0862256e1c42" containerName="main" Jul 08 18:59:28.687382 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.687168 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d2f32a-33a3-45a2-b9d0-0862256e1c42" containerName="main" Jul 08 18:59:28.687382 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.687176 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" containerName="main" Jul 08 18:59:28.687382 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.687182 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" containerName="main" Jul 08 18:59:28.687382 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.687233 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d2f32a-33a3-45a2-b9d0-0862256e1c42" containerName="main" Jul 08 18:59:28.687382 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.687240 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eadcef4c-f1be-4d8c-99a3-1e3bc64ca2c1" containerName="main" Jul 08 18:59:28.690065 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.690050 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.693206 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.693181 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6qtft\"/\"default-dockercfg-4257n\"" Jul 08 18:59:28.693498 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.693482 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6qtft\"/\"openshift-service-ca.crt\"" Jul 08 18:59:28.694038 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.694024 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6qtft\"/\"kube-root-ca.crt\"" Jul 08 18:59:28.707787 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.707764 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql"] Jul 08 18:59:28.750771 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.750732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq54x\" (UniqueName: \"kubernetes.io/projected/2233285b-158f-433d-9a52-6dc5a31d92f7-kube-api-access-gq54x\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.750771 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.750768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-sys\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.750987 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.750797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-proc\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.750987 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.750846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-podres\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.750987 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.750899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-lib-modules\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851710 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-podres\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851882 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-lib-modules\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851882 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq54x\" (UniqueName: \"kubernetes.io/projected/2233285b-158f-433d-9a52-6dc5a31d92f7-kube-api-access-gq54x\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851882 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-sys\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851882 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-proc\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851882 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851834 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-podres\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.851882 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851875 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-sys\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.852164 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851892 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-proc\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.852164 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.851903 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2233285b-158f-433d-9a52-6dc5a31d92f7-lib-modules\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.860006 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.859981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq54x\" (UniqueName: \"kubernetes.io/projected/2233285b-158f-433d-9a52-6dc5a31d92f7-kube-api-access-gq54x\") pod \"perf-node-gather-daemonset-7r5ql\" (UID: \"2233285b-158f-433d-9a52-6dc5a31d92f7\") " pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:28.999501 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:28.999471 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:29.130236 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:29.130139 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql"] Jul 08 18:59:29.132576 ip-10-0-131-106 kubenswrapper[2562]: W0708 18:59:29.132547 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2233285b_158f_433d_9a52_6dc5a31d92f7.slice/crio-2bc0efcf9de22bace0cbcaab5ec0782fd33c2621d368dba32c22a999e2072201 WatchSource:0}: Error finding container 2bc0efcf9de22bace0cbcaab5ec0782fd33c2621d368dba32c22a999e2072201: Status 404 returned error can't find the container with id 2bc0efcf9de22bace0cbcaab5ec0782fd33c2621d368dba32c22a999e2072201 Jul 08 18:59:29.156125 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:29.156095 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" event={"ID":"2233285b-158f-433d-9a52-6dc5a31d92f7","Type":"ContainerStarted","Data":"2bc0efcf9de22bace0cbcaab5ec0782fd33c2621d368dba32c22a999e2072201"} Jul 08 18:59:30.162517 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:30.162480 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" event={"ID":"2233285b-158f-433d-9a52-6dc5a31d92f7","Type":"ContainerStarted","Data":"4218f63b94f3b59b171508a7d014d8367aefae7f3656b004f68c55d6b2f848e5"} Jul 08 18:59:30.162939 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:30.162548 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:30.179596 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:30.179542 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" podStartSLOduration=2.17952406 podStartE2EDuration="2.17952406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:59:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:59:30.17755915 +0000 UTC m=+2483.439624913" watchObservedRunningTime="2026-07-08 18:59:30.17952406 +0000 UTC m=+2483.441589811" Jul 08 18:59:31.247125 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:31.247090 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5gl29_321d20ce-3b8f-4775-9cb6-3f2793a9a235/dns/0.log" Jul 08 18:59:31.268859 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:31.268829 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5gl29_321d20ce-3b8f-4775-9cb6-3f2793a9a235/kube-rbac-proxy/0.log" Jul 08 18:59:31.404586 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:31.404558 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-76cqb_4298d484-fff8-4835-8fdd-d409e875fcaa/dns-node-resolver/0.log" Jul 08 18:59:32.015509 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:32.015473 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5fds_8bfb8749-101a-4ec2-a4ee-19f590ad1350/node-ca/0.log" Jul 08 18:59:33.059505 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:33.059472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb94956b8-l5brt_ce0dcf26-a30f-4c4c-99b0-bd5693a87118/kube-auth-proxy/0.log" Jul 08 18:59:33.985682 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:33.985647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vnlvt_43a6c05e-9764-4048-86c8-4f8aaa99d812/serve-healthcheck-canary/0.log" Jul 08 18:59:34.459190 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:34.459106 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6btwg_cd8fca41-b253-46eb-95cf-554d8657d33d/kube-rbac-proxy/0.log" Jul 08 18:59:34.483182 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:34.483153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6btwg_cd8fca41-b253-46eb-95cf-554d8657d33d/exporter/0.log" Jul 08 18:59:34.505145 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:34.505114 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6btwg_cd8fca41-b253-46eb-95cf-554d8657d33d/extractor/0.log" Jul 08 18:59:36.176710 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:36.176681 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6qtft/perf-node-gather-daemonset-7r5ql" Jul 08 18:59:37.339360 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:37.339321 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-zjf4d_87cde5bd-973f-444b-974c-42ddacccd5e9/manager/0.log" Jul 08 18:59:37.362043 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:37.362016 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-578597fb9c-zjf4d_87cde5bd-973f-444b-974c-42ddacccd5e9/copy-manifests/0.log" Jul 08 18:59:38.819986 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:38.819955 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56665c6f48-9nqb5_34c02218-ccd5-4ba9-9ce7-4b971e963fcf/manager/0.log" Jul 08 18:59:39.326338 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:39.326312 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb94956b8-l5brt_ce0dcf26-a30f-4c4c-99b0-bd5693a87118/kube-auth-proxy/0.log" Jul 08 18:59:40.722326 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:40.722295 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb94956b8-l5brt_ce0dcf26-a30f-4c4c-99b0-bd5693a87118/kube-auth-proxy/0.log" Jul 08 18:59:45.339686 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.339653 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8fqbw_6d774ec5-c014-4267-92bc-9857e91e8dab/kube-multus/0.log" Jul 08 18:59:45.552380 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.552348 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/kube-multus-additional-cni-plugins/0.log" Jul 08 18:59:45.573982 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.573955 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/egress-router-binary-copy/0.log" Jul 08 18:59:45.594337 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.594240 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/cni-plugins/0.log" Jul 08 18:59:45.614731 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.614705 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/bond-cni-plugin/0.log" Jul 08 18:59:45.635407 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.635383 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/routeoverride-cni/0.log" Jul 08 18:59:45.658539 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.658510 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/whereabouts-cni-bincopy/0.log" Jul 08 18:59:45.683468 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.683442 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mjcct_beb08a44-9fdd-42a2-9366-a6bcbeb85cb5/whereabouts-cni/0.log" Jul 08 18:59:45.945429 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.945346 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cgqjt_31aa72aa-4b52-4112-a90a-f8b9f4784008/network-metrics-daemon/0.log" Jul 08 18:59:45.965384 ip-10-0-131-106 kubenswrapper[2562]: I0708 18:59:45.965351 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cgqjt_31aa72aa-4b52-4112-a90a-f8b9f4784008/kube-rbac-proxy/0.log"