Jul 02 14:16:34.455190 ip-10-0-143-249 systemd[1]: Starting Kubernetes Kubelet... Jul 02 14:16:36.135563 ip-10-0-143-249 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 14:16:36.135563 ip-10-0-143-249 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 14:16:36.135563 ip-10-0-143-249 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 14:16:36.135563 ip-10-0-143-249 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 14:16:36.135563 ip-10-0-143-249 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 14:16:36.137761 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.137670 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 14:16:36.141374 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141358 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:16:36.141374 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141374 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141377 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141380 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141383 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141388 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141392 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141395 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141398 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141406 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141409 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141412 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141415 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141417 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141420 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141423 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141425 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141428 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141431 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141433 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141436 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:16:36.141437 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141438 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141441 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141444 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141447 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141449 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141452 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141455 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141457 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141459 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141462 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141465 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141467 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141470 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141474 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141478 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141481 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141484 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141487 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141490 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:16:36.141892 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141492 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141495 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141497 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141500 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141503 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141505 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141508 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141510 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141513 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141516 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141518 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141523 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141526 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141530 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141533 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141536 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141539 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141541 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141544 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:16:36.142392 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141547 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141549 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141552 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141554 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141557 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141560 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141562 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141565 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141567 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141569 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141572 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141574 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141577 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141579 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141582 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141584 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141587 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141590 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141592 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141594 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:16:36.142849 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141597 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141599 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141602 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141604 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141607 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141609 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141612 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141615 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141618 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141620 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141623 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141625 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141628 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141630 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141633 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141636 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141638 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141641 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141644 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.141646 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:16:36.143332 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142405 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142418 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142422 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142425 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142429 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142432 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142435 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142437 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142440 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142443 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142445 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142448 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142451 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142453 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142456 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142458 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142461 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142464 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142466 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142469 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:16:36.143821 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142472 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142474 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142477 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142480 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142482 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142484 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142488 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142492 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142495 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142497 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142500 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142502 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142508 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142512 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142515 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142518 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142520 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142523 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142526 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:16:36.144339 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142528 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142531 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142533 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142536 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142538 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142541 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142543 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142545 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142548 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142552 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142557 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142560 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142563 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142567 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142569 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142572 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142575 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142577 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142580 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142582 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:16:36.144799 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142585 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142588 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142590 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142593 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142595 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142598 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142601 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142604 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142607 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142609 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142612 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142615 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142617 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142621 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142623 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142626 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142628 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142631 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142633 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142635 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:16:36.145295 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142638 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142640 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142643 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142646 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142648 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142651 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142654 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142656 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142659 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142661 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142663 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142666 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142668 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142671 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142673 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142676 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142678 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142681 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142684 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.142687 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:16:36.145775 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143048 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143052 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143055 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143058 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143061 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143063 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143066 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143069 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143071 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143074 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143076 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143079 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143082 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143085 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143088 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143090 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143093 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143095 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143098 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143101 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:16:36.146264 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143103 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143106 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143108 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143111 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143113 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143116 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143119 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143122 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143124 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143127 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143129 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143132 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143135 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143138 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143140 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143143 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143148 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143166 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143169 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143172 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:16:36.146757 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143174 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143177 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143180 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143182 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143187 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143190 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143194 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143197 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143200 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143202 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143205 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143208 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143210 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143213 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143217 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143221 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143224 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143226 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143228 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:16:36.147248 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143231 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143234 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143237 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143239 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143242 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143244 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143248 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143251 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143253 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143257 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143260 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143262 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143265 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143268 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143270 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143272 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143275 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143277 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143280 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143282 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:16:36.147708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143285 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143287 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143290 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143293 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143295 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143298 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143300 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143304 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143307 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143310 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143313 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143316 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143319 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143322 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143325 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143329 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143331 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143334 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143337 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.143340 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:16:36.148255 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143412 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143422 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143429 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143434 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143438 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143442 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143446 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143451 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143454 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143458 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143462 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143465 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143468 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143472 2581 flags.go:64] FLAG: --cgroup-root="" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143475 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143478 2581 flags.go:64] FLAG: --client-ca-file="" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143481 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143484 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143488 2581 flags.go:64] FLAG: --cluster-domain="" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143491 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143494 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143497 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143501 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143504 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 14:16:36.148731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143507 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143510 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143514 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143517 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143520 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143523 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143526 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143529 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143534 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143538 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143558 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143562 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143566 2581 flags.go:64] FLAG: --enable-server="true" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143569 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143575 2581 flags.go:64] FLAG: --event-burst="100" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143578 2581 flags.go:64] FLAG: --event-qps="50" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143581 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143584 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143587 2581 flags.go:64] FLAG: --eviction-hard="" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143591 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143594 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143597 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143600 2581 flags.go:64] FLAG: --eviction-soft="" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143603 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143606 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 14:16:36.149311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143609 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143612 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143614 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143617 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143620 2581 flags.go:64] FLAG: --feature-gates="" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143624 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143626 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143631 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143634 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143637 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143640 2581 flags.go:64] FLAG: --help="false" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143643 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143646 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143649 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143652 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143656 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143661 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143664 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143667 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143670 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143673 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143676 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143679 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143682 2581 flags.go:64] FLAG: --kube-reserved="" Jul 02 14:16:36.150076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143685 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143688 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143691 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143694 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143697 2581 flags.go:64] FLAG: --lock-file="" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143700 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143703 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143706 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143712 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143715 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143717 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143720 2581 flags.go:64] FLAG: --logging-format="text" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143723 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143726 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143729 2581 flags.go:64] FLAG: --manifest-url="" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143735 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143739 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143742 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143746 2581 flags.go:64] FLAG: --max-pods="110" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143749 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143752 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143755 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143758 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143761 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143765 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 14:16:36.150698 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143769 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143776 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143779 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143782 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143785 2581 flags.go:64] FLAG: --pod-cidr="" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143788 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143793 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143797 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143800 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143803 2581 flags.go:64] FLAG: --port="10250" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143806 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143809 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01d326a6f46b29775" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143812 2581 flags.go:64] FLAG: --qos-reserved="" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143815 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143818 2581 flags.go:64] FLAG: --register-node="true" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143821 2581 flags.go:64] FLAG: --register-with-taints="" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143825 2581 flags.go:64] FLAG: --registry-burst="10" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143828 2581 flags.go:64] FLAG: --registry-qps="5" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143831 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143834 2581 flags.go:64] FLAG: --reserved-memory="" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143837 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143841 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143844 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143848 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143851 2581 flags.go:64] FLAG: --runonce="false" Jul 02 14:16:36.151569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143854 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143857 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143860 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143863 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143865 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143869 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143873 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143876 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143880 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143883 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143886 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143889 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143892 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143895 2581 flags.go:64] FLAG: --system-cgroups="" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143898 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143904 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143907 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143910 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143914 2581 flags.go:64] FLAG: --tls-min-version="" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143916 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143919 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143922 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143925 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143928 2581 flags.go:64] FLAG: --v="2" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143932 2581 flags.go:64] FLAG: --version="false" Jul 02 14:16:36.152339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143936 2581 flags.go:64] FLAG: --vmodule="" Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143939 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.143943 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144059 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144063 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144067 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144070 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144073 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144075 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144078 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144080 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144083 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144085 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144090 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144092 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144095 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144098 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144101 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144103 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144106 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:16:36.152963 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144108 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144111 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144114 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144116 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144119 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144122 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144124 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144127 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144129 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144132 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144134 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144137 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144139 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144142 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144144 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144147 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144168 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144182 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144185 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144188 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:16:36.153491 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144191 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144194 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144196 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144199 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144202 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144205 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144208 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144210 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144213 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144216 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144219 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144221 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144224 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144228 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144231 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144234 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144237 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144239 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144242 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:16:36.153969 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144245 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144247 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144250 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144252 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144255 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144257 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144260 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144262 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144265 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144268 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144272 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144275 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144277 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144280 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144282 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144285 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144287 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144289 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144293 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144296 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:16:36.154431 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144299 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144301 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144304 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144307 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144309 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144312 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144316 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144319 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144322 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144325 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144327 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144330 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144332 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144335 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144337 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144340 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144342 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144345 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144347 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144350 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:16:36.154907 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144353 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144355 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.144360 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.144366 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153000 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153019 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153040 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153050 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153109 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153114 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153117 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153120 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153124 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153127 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153130 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:16:36.155393 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153132 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153135 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153138 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153140 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153143 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153146 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153148 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153163 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153166 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153169 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153171 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153174 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153177 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153180 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153183 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153185 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153188 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153191 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153194 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:16:36.155763 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153197 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153199 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153202 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153206 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153209 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153211 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153214 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153217 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153219 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153222 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153224 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153227 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153230 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153232 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153235 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153238 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153240 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153243 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153247 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:16:36.156239 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153251 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153254 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153257 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153260 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153263 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153265 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153268 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153270 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153273 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153275 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153278 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153281 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153283 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153286 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153289 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153291 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153295 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153298 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153302 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:16:36.156694 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153306 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153309 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153311 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153314 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153317 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153319 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153322 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153324 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153327 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153330 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153334 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153337 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153339 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153342 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153344 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153347 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153350 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153352 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153355 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153358 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:16:36.157186 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153360 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153363 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153365 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153368 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153370 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153373 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153375 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153378 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153381 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153384 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153386 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153389 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153392 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153394 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153397 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:16:36.157665 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153402 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153523 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153527 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153531 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153534 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153537 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153540 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153543 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153546 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153549 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153551 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153554 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153557 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153560 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153563 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153565 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153568 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153570 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153573 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153577 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 14:16:36.158035 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153579 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153582 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153585 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153588 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153591 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153593 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153596 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153599 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153603 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153606 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153609 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153612 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153615 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153618 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153620 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153623 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153625 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153628 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153630 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153633 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 14:16:36.158526 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153635 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153638 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153641 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153643 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153646 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153648 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153651 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153654 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153657 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153659 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153662 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153664 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153667 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153670 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153672 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153675 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153677 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153680 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153683 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153685 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 14:16:36.159010 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153688 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153690 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153693 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153695 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153698 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153700 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153702 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153705 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153707 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153709 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153712 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153714 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153717 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153719 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153721 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153724 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153726 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153729 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153731 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153734 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 14:16:36.159498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153736 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153739 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153741 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153745 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153748 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153750 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153753 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153756 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153758 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153761 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153764 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153767 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153769 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153772 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153774 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153777 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153779 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153782 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153784 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 14:16:36.159958 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:36.153786 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 14:16:36.160441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153792 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 14:16:36.160441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.153927 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 14:16:36.160441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.156800 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 14:16:36.160441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.158122 2581 server.go:1019] "Starting client certificate rotation" Jul 02 14:16:36.160441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.158189 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 14:16:36.160441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.158225 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 14:16:36.160597 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.160476 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 14:16:36.161664 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.161644 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 14:16:36.171108 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.171092 2581 log.go:25] "Validated CRI v1 runtime API" Jul 02 14:16:36.173065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.173046 2581 log.go:25] "Validated CRI v1 image API" Jul 02 14:16:36.173895 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.173875 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 14:16:36.176092 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.176073 2581 fs.go:135] Filesystem UUIDs: map[6793bdda-5b9e-4ec7-b207-d61122c7bf48:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 afe445bc-03bd-4696-906b-10d21196257c:/dev/nvme0n1p3] Jul 02 14:16:36.176142 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.176092 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 14:16:36.181437 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.181328 2581 manager.go:216] Machine: {Timestamp:2026-07-02 14:16:36.180052073 +0000 UTC m=+0.301294303 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098942 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec247445088b66c622be53fc3ad2a32e SystemUUID:ec247445-088b-66c6-22be-53fc3ad2a32e BootID:75450f57-f9c8-48e1-b867-c72906e52a93 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:02:83:e3:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:02:83:e3:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:2e:7a:e6:76:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 14:16:36.181437 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.181435 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 14:16:36.181557 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.181519 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 14:16:36.184118 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.184091 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 14:16:36.184285 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.184121 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-249.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 14:16:36.185714 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.185702 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 14:16:36.185750 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.185717 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 14:16:36.185750 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.185724 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 14:16:36.185750 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.185738 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 14:16:36.186510 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.186499 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 14:16:36.189328 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.189317 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 02 14:16:36.189450 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.189440 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 14:16:36.190110 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.190101 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 02 14:16:36.190141 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.190115 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 14:16:36.190141 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.190134 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 14:16:36.190203 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.190145 2581 kubelet.go:398] "Adding apiserver pod source" Jul 02 14:16:36.190203 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.190169 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 14:16:36.191418 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.191383 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 14:16:36.191418 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.191415 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 14:16:36.195499 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.195481 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 14:16:36.196839 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.196826 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 14:16:36.196900 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.196856 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 14:16:36.197045 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197035 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 14:16:36.197076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197050 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 14:16:36.197076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197056 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 14:16:36.197076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197062 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 14:16:36.197076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197069 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 14:16:36.197076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197075 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 14:16:36.197498 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197489 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 14:16:36.197498 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197498 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 14:16:36.197558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197505 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 14:16:36.197558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197510 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 14:16:36.197558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197519 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 14:16:36.197558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.197527 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 14:16:36.198100 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.198086 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 14:16:36.198100 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.198097 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 14:16:36.198343 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.198318 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 14:16:36.200216 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.200196 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-249.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 14:16:36.200883 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.200859 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 14:16:36.200943 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.200859 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 14:16:36.201459 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.201445 2581 server.go:1268] "Started kubelet" Jul 02 14:16:36.201581 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.201531 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 14:16:36.202269 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.202110 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 14:16:36.202269 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.202205 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 14:16:36.202425 ip-10-0-143-249 systemd[1]: Started Kubernetes Kubelet. Jul 02 14:16:36.203783 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.203769 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 02 14:16:36.205653 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.205636 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 14:16:36.207582 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.207551 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 14:16:36.211061 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211046 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 14:16:36.211131 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211065 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 14:16:36.211206 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.211144 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.211694 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211675 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 14:16:36.211694 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211677 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 14:16:36.211833 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211703 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 14:16:36.211833 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.210894 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-249.ec2.internal.18be7eafd878098b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-249.ec2.internal,UID:ip-10-0-143-249.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-249.ec2.internal,},FirstTimestamp:2026-07-02 14:16:36.201417099 +0000 UTC m=+0.322659331,LastTimestamp:2026-07-02 14:16:36.201417099 +0000 UTC m=+0.322659331,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-249.ec2.internal,}" Jul 02 14:16:36.211833 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211810 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 02 14:16:36.211833 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.211820 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 02 14:16:36.214058 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214043 2581 factory.go:55] Registering systemd factory Jul 02 14:16:36.214058 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214059 2581 factory.go:223] Registration of the systemd container factory successfully Jul 02 14:16:36.214434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214417 2581 factory.go:153] Registering CRI-O factory Jul 02 14:16:36.214434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214434 2581 factory.go:223] Registration of the crio container factory successfully Jul 02 14:16:36.214541 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214475 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 14:16:36.214541 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214497 2581 factory.go:103] Registering Raw factory Jul 02 14:16:36.214541 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.214510 2581 manager.go:1196] Started watching for new ooms in manager Jul 02 14:16:36.215315 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.215302 2581 manager.go:349] Starting recovery of all containers Jul 02 14:16:36.220046 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.220019 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gzbv5" Jul 02 14:16:36.226006 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.225973 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 14:16:36.226132 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.226091 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-249.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 14:16:36.228325 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.228303 2581 manager.go:354] Recovery completed Jul 02 14:16:36.228823 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.228803 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gzbv5" Jul 02 14:16:36.232498 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.232485 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.234724 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.234704 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.234826 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.234744 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.234826 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.234757 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.235386 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.235370 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 14:16:36.235386 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.235381 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.235464 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.235387 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 14:16:36.235464 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.235407 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 02 14:16:36.236096 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.236081 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.236141 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.236107 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.236141 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.236116 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.236653 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.236637 2581 policy_none.go:49] "None policy: Start" Jul 02 14:16:36.236714 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.236660 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 14:16:36.236714 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.236675 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 14:16:36.238038 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.238026 2581 policy_none.go:47] "Start" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.270925 2581 manager.go:337] "Starting Device Plugin manager" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.271176 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.271190 2581 server.go:87] "Starting device plugin registration server" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.271473 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.271486 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.271604 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.271693 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.271702 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.272214 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.272270 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.278132 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.279329 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.279365 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.279385 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.279418 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 14:16:36.282853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.281923 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:16:36.372280 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.372189 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.373330 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.373314 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.373429 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.373347 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.373429 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.373363 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.373429 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.373389 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.380269 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.380245 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal"] Jul 02 14:16:36.380330 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.380320 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.382615 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.382600 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.382690 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.382624 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.382690 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.382633 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.383867 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.383856 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.383978 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.383966 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.384058 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.383996 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.384553 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.384536 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.384625 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.384563 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.384625 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.384592 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.384625 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.384593 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.384625 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.384612 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.384739 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.384625 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.385858 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.385841 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.385958 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.385868 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 14:16:36.386019 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.385960 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.386019 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.385981 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-249.ec2.internal\": node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.386540 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.386524 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientMemory" Jul 02 14:16:36.386613 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.386555 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 14:16:36.386613 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.386569 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeHasSufficientPID" Jul 02 14:16:36.405925 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.405903 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.409772 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.409755 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-249.ec2.internal\" not found" node="ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.412372 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.412355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6477d183aafb4ad9f6bb122000884dcf-config\") pod \"kube-apiserver-proxy-ip-10-0-143-249.ec2.internal\" (UID: \"6477d183aafb4ad9f6bb122000884dcf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.412452 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.412379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9e3f640dc2f40324fd180adbbfd0129f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal\" (UID: \"9e3f640dc2f40324fd180adbbfd0129f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.412452 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.412398 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e3f640dc2f40324fd180adbbfd0129f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal\" (UID: \"9e3f640dc2f40324fd180adbbfd0129f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.413192 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.413176 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-249.ec2.internal\" not found" node="ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.506654 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.506612 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.512896 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.512867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6477d183aafb4ad9f6bb122000884dcf-config\") pod \"kube-apiserver-proxy-ip-10-0-143-249.ec2.internal\" (UID: \"6477d183aafb4ad9f6bb122000884dcf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.512896 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.512901 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9e3f640dc2f40324fd180adbbfd0129f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal\" (UID: \"9e3f640dc2f40324fd180adbbfd0129f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.513004 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.512921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e3f640dc2f40324fd180adbbfd0129f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal\" (UID: \"9e3f640dc2f40324fd180adbbfd0129f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.513004 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.512966 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6477d183aafb4ad9f6bb122000884dcf-config\") pod \"kube-apiserver-proxy-ip-10-0-143-249.ec2.internal\" (UID: \"6477d183aafb4ad9f6bb122000884dcf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.513082 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.513015 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e3f640dc2f40324fd180adbbfd0129f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal\" (UID: \"9e3f640dc2f40324fd180adbbfd0129f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.513082 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.513040 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9e3f640dc2f40324fd180adbbfd0129f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal\" (UID: \"9e3f640dc2f40324fd180adbbfd0129f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.606990 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.606955 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.707274 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.707208 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.712374 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.712354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.715368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:36.715355 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" Jul 02 14:16:36.807902 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.807863 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:36.908397 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:36.908361 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.008976 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.008904 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.109481 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.109449 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.157965 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.157936 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 14:16:37.158450 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.158101 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 14:16:37.209603 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.209575 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.232781 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.232746 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 14:11:36 +0000 UTC" deadline="2028-04-18 06:37:43.666998838 +0000 UTC" Jul 02 14:16:37.232781 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.232775 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15736h21m6.434226595s" Jul 02 14:16:37.249549 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.249526 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:16:37.254527 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:37.254498 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6477d183aafb4ad9f6bb122000884dcf.slice/crio-d228308ac196f26ad11c1e6a46d968ace4157a1da1976848fd488dca3b825e4b WatchSource:0}: Error finding container d228308ac196f26ad11c1e6a46d968ace4157a1da1976848fd488dca3b825e4b: Status 404 returned error can't find the container with id d228308ac196f26ad11c1e6a46d968ace4157a1da1976848fd488dca3b825e4b Jul 02 14:16:37.260801 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.260782 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 14:16:37.283230 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.283180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" event={"ID":"6477d183aafb4ad9f6bb122000884dcf","Type":"ContainerStarted","Data":"d228308ac196f26ad11c1e6a46d968ace4157a1da1976848fd488dca3b825e4b"} Jul 02 14:16:37.310317 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.310289 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.359662 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:37.359636 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3f640dc2f40324fd180adbbfd0129f.slice/crio-ab883b0623c7f1e5dbc5b5295803db42f85635af1a27a87e8c4ae50adcbbbe88 WatchSource:0}: Error finding container ab883b0623c7f1e5dbc5b5295803db42f85635af1a27a87e8c4ae50adcbbbe88: Status 404 returned error can't find the container with id ab883b0623c7f1e5dbc5b5295803db42f85635af1a27a87e8c4ae50adcbbbe88 Jul 02 14:16:37.410992 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.410955 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.511449 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.511380 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.611835 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:37.611804 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-249.ec2.internal\" not found" Jul 02 14:16:37.668883 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.668859 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:16:37.712148 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.712109 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" Jul 02 14:16:37.722354 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.722320 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 14:16:37.724304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.724289 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" Jul 02 14:16:37.733597 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.733571 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 14:16:37.756892 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:37.756860 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:16:38.191623 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.191584 2581 apiserver.go:52] "Watching apiserver" Jul 02 14:16:38.199249 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.199224 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 14:16:38.199604 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.199581 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dbr7t","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r","openshift-image-registry/node-ca-kd2r7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal","openshift-multus/multus-9fsww","openshift-multus/multus-additional-cni-plugins-2rxvc","openshift-multus/network-metrics-daemon-4mhz9","openshift-network-diagnostics/network-check-target-8vhdd","kube-system/konnectivity-agent-6qkws","kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal","openshift-cluster-node-tuning-operator/tuned-c5vtl","openshift-dns/node-resolver-fsmwf","openshift-network-operator/iptables-alerter-qpqhq","openshift-ovn-kubernetes/ovnkube-node-m7grd"] Jul 02 14:16:38.200943 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.200928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.202063 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.202043 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.204892 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.204861 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205501 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205509 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205537 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205580 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205602 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6j9c9\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205646 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205540 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 14:16:38.205771 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205661 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n2xxv\"" Jul 02 14:16:38.206078 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.205874 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.206078 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.205963 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:38.206929 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.206910 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.207883 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.207864 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 14:16:38.208185 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.208147 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vtdsj\"" Jul 02 14:16:38.208274 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.208240 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.208600 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.208586 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.208695 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.208682 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.209040 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.209024 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-58zgp\"" Jul 02 14:16:38.209220 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.209202 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 14:16:38.209301 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.209237 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.209418 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.209406 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.209465 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.209419 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 14:16:38.210189 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.210050 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.210189 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.210121 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:38.210680 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.210659 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bm92k\"" Jul 02 14:16:38.211255 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.210801 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 14:16:38.211255 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.211004 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 14:16:38.211859 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.211844 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 14:16:38.211954 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.211939 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:38.212014 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.211994 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:38.213396 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.213377 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.215494 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.215477 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.215875 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.215626 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 14:16:38.215875 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.215732 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 14:16:38.215875 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.215763 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n4cff\"" Jul 02 14:16:38.217553 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.217522 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.217697 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.217677 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.218003 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.217974 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 14:16:38.218123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.218054 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.218438 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.218287 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 14:16:38.218438 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.218333 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-66drc\"" Jul 02 14:16:38.218438 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.218424 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 14:16:38.219061 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219044 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.219274 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219254 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-iptables-alerter-script\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.219380 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219287 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-etc-kubernetes\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.219380 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219324 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl4lx\" (UniqueName: \"kubernetes.io/projected/a8ec2efe-70d4-4636-9f1a-914333629928-kube-api-access-jl4lx\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.219380 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.219546 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219388 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-host-slash\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.219546 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219417 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-cnibin\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.219546 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql7mw\" (UniqueName: \"kubernetes.io/projected/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-kube-api-access-ql7mw\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.219546 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219481 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 14:16:38.219546 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz29b\" (UniqueName: \"kubernetes.io/projected/c468cb3a-4a0d-48db-8aae-80b251657d8a-kube-api-access-rz29b\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219566 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-682wv\" (UniqueName: \"kubernetes.io/projected/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-kube-api-access-682wv\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219593 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-netns\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219622 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-cni-multus\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-hostroot\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-system-cni-dir\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219740 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-sys-fs\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.219797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219777 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvn5p\" (UniqueName: \"kubernetes.io/projected/4911423d-9c72-42af-b0e0-73194c3f1c5e-kube-api-access-xvn5p\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-kubelet-config\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-system-cni-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219846 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-cnibin\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219870 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-os-release\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219919 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219952 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-socket-dir-parent\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.219984 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-registration-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220007 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220007 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220011 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xntzp\" (UniqueName: \"kubernetes.io/projected/2bffbb99-79da-4149-8564-b3b77a67615f-kube-api-access-xntzp\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-kubelet\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220142 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-conf-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.220279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220222 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-k8s-cni-cncf-io\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-cni-bin\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220275 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2efe-70d4-4636-9f1a-914333629928-multus-daemon-config\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220283 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s6qtk\"" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-multus-certs\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8ec2efe-70d4-4636-9f1a-914333629928-cni-binary-copy\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-socket-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220793 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-device-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2bffbb99-79da-4149-8564-b3b77a67615f-host\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2bffbb99-79da-4149-8564-b3b77a67615f-serviceca\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220880 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-dbus\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-cni-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.221071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.220968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-os-release\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.222110 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.221444 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kcc7b\"" Jul 02 14:16:38.222110 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.221468 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 14:16:38.222709 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.222689 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 14:16:38.229043 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.229023 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 14:16:38.253047 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.253026 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nmz72" Jul 02 14:16:38.260782 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.260762 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nmz72" Jul 02 14:16:38.285869 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.285838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" event={"ID":"9e3f640dc2f40324fd180adbbfd0129f","Type":"ContainerStarted","Data":"ab883b0623c7f1e5dbc5b5295803db42f85635af1a27a87e8c4ae50adcbbbe88"} Jul 02 14:16:38.312768 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.312745 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 14:16:38.321775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321741 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-cni-multus\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.321775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-hostroot\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321795 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-modprobe-d\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321854 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-cni-multus\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321867 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-hostroot\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-systemd\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-var-lib-kubelet\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.321984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321971 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-sys-fs\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.321996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-kubelet-config\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322011 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-systemd\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322031 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/545cf0b4-dd6f-489f-8f52-3c697db25831-tmp-dir\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322035 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-sys-fs\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-socket-dir-parent\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-registration-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322089 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-kubelet-config\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322116 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xntzp\" (UniqueName: \"kubernetes.io/projected/2bffbb99-79da-4149-8564-b3b77a67615f-kube-api-access-xntzp\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322220 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-socket-dir-parent\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322302 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-ovnkube-config\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysctl-conf\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322387 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-registration-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15be550e-4122-4920-bc22-669d9b336248-etc-tuned\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322436 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-cni-binary-copy\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/545cf0b4-dd6f-489f-8f52-3c697db25831-hosts-file\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-conf-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-log-socket\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-k8s-cni-cncf-io\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-cni-bin\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322594 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-conf-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-k8s-cni-cncf-io\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-systemd-units\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322685 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-cni-bin\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-env-overrides\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysconfig\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.322834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322789 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8ec2efe-70d4-4636-9f1a-914333629928-cni-binary-copy\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322814 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-socket-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.322916 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2bffbb99-79da-4149-8564-b3b77a67615f-serviceca\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323008 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-dbus\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323006 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-socket-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl4lx\" (UniqueName: \"kubernetes.io/projected/a8ec2efe-70d4-4636-9f1a-914333629928-kube-api-access-jl4lx\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323171 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b41bd1d-1f33-40b9-95a0-2347fa038e15-konnectivity-ca\") pod \"konnectivity-agent-6qkws\" (UID: \"9b41bd1d-1f33-40b9-95a0-2347fa038e15\") " pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323212 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-ovn\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323219 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-dbus\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323234 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-slash\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-run-netns\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323293 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-etc-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323311 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8ec2efe-70d4-4636-9f1a-914333629928-cni-binary-copy\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-node-log\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323343 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-cni-netd\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323375 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7cg2\" (UniqueName: \"kubernetes.io/projected/15be550e-4122-4920-bc22-669d9b336248-kube-api-access-z7cg2\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323390 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2bffbb99-79da-4149-8564-b3b77a67615f-serviceca\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.323567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql7mw\" (UniqueName: \"kubernetes.io/projected/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-kube-api-access-ql7mw\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz29b\" (UniqueName: \"kubernetes.io/projected/c468cb3a-4a0d-48db-8aae-80b251657d8a-kube-api-access-rz29b\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-netns\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-system-cni-dir\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323571 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323586 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-netns\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323602 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-sys\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323687 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-system-cni-dir\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323765 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvn5p\" (UniqueName: \"kubernetes.io/projected/4911423d-9c72-42af-b0e0-73194c3f1c5e-kube-api-access-xvn5p\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323791 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-system-cni-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-cnibin\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323864 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-os-release\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323909 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-cnibin\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323967 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c468cb3a-4a0d-48db-8aae-80b251657d8a-os-release\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323984 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-cni-bin\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.323913 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-system-cni-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324333 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-kubelet\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324375 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-kubelet\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324400 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysctl-d\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324451 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-lib-modules\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c468cb3a-4a0d-48db-8aae-80b251657d8a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324468 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-etc-selinux\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-var-lib-kubelet\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324543 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-host\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324566 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-ovnkube-script-lib\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324616 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324650 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.324997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324700 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2efe-70d4-4636-9f1a-914333629928-multus-daemon-config\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-multus-certs\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr6pm\" (UniqueName: \"kubernetes.io/projected/c5630e44-468e-4982-b742-d7b34aea53d2-kube-api-access-sr6pm\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324791 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-host-run-multus-certs\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.324853 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.324864 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvtv\" (UniqueName: \"kubernetes.io/projected/545cf0b4-dd6f-489f-8f52-3c697db25831-kube-api-access-7cvtv\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.324967 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:38.824912558 +0000 UTC m=+2.946154787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325001 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-device-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2bffbb99-79da-4149-8564-b3b77a67615f-host\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-cni-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325074 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325144 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8ec2efe-70d4-4636-9f1a-914333629928-multus-daemon-config\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325101 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4911423d-9c72-42af-b0e0-73194c3f1c5e-device-dir\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325103 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2bffbb99-79da-4149-8564-b3b77a67615f-host\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325193 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b41bd1d-1f33-40b9-95a0-2347fa038e15-agent-certs\") pod \"konnectivity-agent-6qkws\" (UID: \"9b41bd1d-1f33-40b9-95a0-2347fa038e15\") " pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.325265 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325143 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-multus-cni-dir\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.325786 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.325401 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:38.825363433 +0000 UTC m=+2.946605655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325713 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-run-ovn-kubernetes\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5630e44-468e-4982-b742-d7b34aea53d2-ovn-node-metrics-cert\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325775 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-var-lib-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325802 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-os-release\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-iptables-alerter-script\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325868 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-os-release\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-etc-kubernetes\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.325960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-kubernetes\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-run\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326024 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-etc-kubernetes\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326049 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15be550e-4122-4920-bc22-669d9b336248-tmp\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-host-slash\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326146 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-cnibin\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326175 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-host-slash\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326192 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-682wv\" (UniqueName: \"kubernetes.io/projected/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-kube-api-access-682wv\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326239 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8ec2efe-70d4-4636-9f1a-914333629928-cnibin\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.326457 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.326375 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-iptables-alerter-script\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.329250 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.329076 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 14:16:38.333400 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.333380 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xntzp\" (UniqueName: \"kubernetes.io/projected/2bffbb99-79da-4149-8564-b3b77a67615f-kube-api-access-xntzp\") pod \"node-ca-kd2r7\" (UID: \"2bffbb99-79da-4149-8564-b3b77a67615f\") " pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.333718 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.333693 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:38.333718 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.333718 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:38.333852 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.333731 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:38.333852 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.333780 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:16:38.833763591 +0000 UTC m=+2.955005824 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:38.336065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.336042 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl4lx\" (UniqueName: \"kubernetes.io/projected/a8ec2efe-70d4-4636-9f1a-914333629928-kube-api-access-jl4lx\") pod \"multus-9fsww\" (UID: \"a8ec2efe-70d4-4636-9f1a-914333629928\") " pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.336449 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.336425 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvn5p\" (UniqueName: \"kubernetes.io/projected/4911423d-9c72-42af-b0e0-73194c3f1c5e-kube-api-access-xvn5p\") pod \"aws-ebs-csi-driver-node-8tw4r\" (UID: \"4911423d-9c72-42af-b0e0-73194c3f1c5e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.337144 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.337090 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz29b\" (UniqueName: \"kubernetes.io/projected/c468cb3a-4a0d-48db-8aae-80b251657d8a-kube-api-access-rz29b\") pod \"multus-additional-cni-plugins-2rxvc\" (UID: \"c468cb3a-4a0d-48db-8aae-80b251657d8a\") " pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.337615 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.337573 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql7mw\" (UniqueName: \"kubernetes.io/projected/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-kube-api-access-ql7mw\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.338305 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.338283 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-682wv\" (UniqueName: \"kubernetes.io/projected/5fdfa295-ce5b-40f8-bb82-71d6ec26a879-kube-api-access-682wv\") pod \"iptables-alerter-qpqhq\" (UID: \"5fdfa295-ce5b-40f8-bb82-71d6ec26a879\") " pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.412419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.412389 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 14:16:38.427136 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427110 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-systemd-units\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427136 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-env-overrides\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysconfig\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427193 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b41bd1d-1f33-40b9-95a0-2347fa038e15-konnectivity-ca\") pod \"konnectivity-agent-6qkws\" (UID: \"9b41bd1d-1f33-40b9-95a0-2347fa038e15\") " pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-ovn\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427231 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-systemd-units\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427259 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-ovn\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427272 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysconfig\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.427356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427326 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-slash\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427364 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-slash\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-run-netns\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427428 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-etc-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-node-log\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-cni-netd\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427483 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-run-netns\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427533 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-node-log\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427544 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-cni-netd\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427494 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-etc-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7cg2\" (UniqueName: \"kubernetes.io/projected/15be550e-4122-4920-bc22-669d9b336248-kube-api-access-z7cg2\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-sys\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427666 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-cni-bin\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427705 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427713 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-env-overrides\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.427730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-kubelet\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysctl-d\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427755 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-cni-bin\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427767 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-lib-modules\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427772 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427789 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-host\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427776 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b41bd1d-1f33-40b9-95a0-2347fa038e15-konnectivity-ca\") pod \"konnectivity-agent-6qkws\" (UID: \"9b41bd1d-1f33-40b9-95a0-2347fa038e15\") " pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427806 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-sys\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427815 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-kubelet\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427815 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-ovnkube-script-lib\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427850 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-host\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr6pm\" (UniqueName: \"kubernetes.io/projected/c5630e44-468e-4982-b742-d7b34aea53d2-kube-api-access-sr6pm\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427919 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvtv\" (UniqueName: \"kubernetes.io/projected/545cf0b4-dd6f-489f-8f52-3c697db25831-kube-api-access-7cvtv\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427921 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysctl-d\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427947 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-lib-modules\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427957 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b41bd1d-1f33-40b9-95a0-2347fa038e15-agent-certs\") pod \"konnectivity-agent-6qkws\" (UID: \"9b41bd1d-1f33-40b9-95a0-2347fa038e15\") " pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.427982 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-run-ovn-kubernetes\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.428503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428055 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5630e44-468e-4982-b742-d7b34aea53d2-ovn-node-metrics-cert\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428096 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-host-run-ovn-kubernetes\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428096 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-var-lib-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-var-lib-openvswitch\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428146 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-kubernetes\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428198 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-run\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428221 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15be550e-4122-4920-bc22-669d9b336248-tmp\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-modprobe-d\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428255 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-kubernetes\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-systemd\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-var-lib-kubelet\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428325 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-run\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-systemd\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428363 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/545cf0b4-dd6f-489f-8f52-3c697db25831-tmp-dir\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-ovnkube-config\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428406 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-ovnkube-script-lib\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysctl-conf\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428435 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-modprobe-d\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.429304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428457 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15be550e-4122-4920-bc22-669d9b336248-etc-tuned\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/545cf0b4-dd6f-489f-8f52-3c697db25831-hosts-file\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428495 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-var-lib-kubelet\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428473 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-systemd\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428364 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-run-systemd\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428512 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-log-socket\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428567 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/545cf0b4-dd6f-489f-8f52-3c697db25831-hosts-file\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428641 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c5630e44-468e-4982-b742-d7b34aea53d2-log-socket\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428689 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/15be550e-4122-4920-bc22-669d9b336248-etc-sysctl-conf\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.428885 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/545cf0b4-dd6f-489f-8f52-3c697db25831-tmp-dir\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.430025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.429001 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c5630e44-468e-4982-b742-d7b34aea53d2-ovnkube-config\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.430813 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.430791 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b41bd1d-1f33-40b9-95a0-2347fa038e15-agent-certs\") pod \"konnectivity-agent-6qkws\" (UID: \"9b41bd1d-1f33-40b9-95a0-2347fa038e15\") " pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.431102 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.431071 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c5630e44-468e-4982-b742-d7b34aea53d2-ovn-node-metrics-cert\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.431198 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.431108 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/15be550e-4122-4920-bc22-669d9b336248-etc-tuned\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.431297 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.431282 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15be550e-4122-4920-bc22-669d9b336248-tmp\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.441760 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.441703 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvtv\" (UniqueName: \"kubernetes.io/projected/545cf0b4-dd6f-489f-8f52-3c697db25831-kube-api-access-7cvtv\") pod \"node-resolver-fsmwf\" (UID: \"545cf0b4-dd6f-489f-8f52-3c697db25831\") " pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.441912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.441895 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7cg2\" (UniqueName: \"kubernetes.io/projected/15be550e-4122-4920-bc22-669d9b336248-kube-api-access-z7cg2\") pod \"tuned-c5vtl\" (UID: \"15be550e-4122-4920-bc22-669d9b336248\") " pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.441965 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.441910 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr6pm\" (UniqueName: \"kubernetes.io/projected/c5630e44-468e-4982-b742-d7b34aea53d2-kube-api-access-sr6pm\") pod \"ovnkube-node-m7grd\" (UID: \"c5630e44-468e-4982-b742-d7b34aea53d2\") " pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.514862 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.514817 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qpqhq" Jul 02 14:16:38.521824 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.521801 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" Jul 02 14:16:38.522544 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.522522 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdfa295_ce5b_40f8_bb82_71d6ec26a879.slice/crio-edff550543193b696d04b1567101fc31c8298c42fe2fa19fe698e6102cfaf45e WatchSource:0}: Error finding container edff550543193b696d04b1567101fc31c8298c42fe2fa19fe698e6102cfaf45e: Status 404 returned error can't find the container with id edff550543193b696d04b1567101fc31c8298c42fe2fa19fe698e6102cfaf45e Jul 02 14:16:38.531080 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.531062 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kd2r7" Jul 02 14:16:38.535769 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.535754 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9fsww" Jul 02 14:16:38.541960 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.541935 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4911423d_9c72_42af_b0e0_73194c3f1c5e.slice/crio-22c3696392d0df08122f07d494fb66e67d0338d1ae1ebf30d603e6deea9aaa9b WatchSource:0}: Error finding container 22c3696392d0df08122f07d494fb66e67d0338d1ae1ebf30d603e6deea9aaa9b: Status 404 returned error can't find the container with id 22c3696392d0df08122f07d494fb66e67d0338d1ae1ebf30d603e6deea9aaa9b Jul 02 14:16:38.542630 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.542613 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" Jul 02 14:16:38.548826 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.548806 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:16:38.550056 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.550032 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bffbb99_79da_4149_8564_b3b77a67615f.slice/crio-1ea320829ff6d7ac59257650bd46f7e9bce68ad3d28a1fdd62664e095307fe8d WatchSource:0}: Error finding container 1ea320829ff6d7ac59257650bd46f7e9bce68ad3d28a1fdd62664e095307fe8d: Status 404 returned error can't find the container with id 1ea320829ff6d7ac59257650bd46f7e9bce68ad3d28a1fdd62664e095307fe8d Jul 02 14:16:38.550568 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.550546 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ec2efe_70d4_4636_9f1a_914333629928.slice/crio-71b4855d5557e9dfe4182843bc43910a7a72ed9cd158119ca005e49d7180e347 WatchSource:0}: Error finding container 71b4855d5557e9dfe4182843bc43910a7a72ed9cd158119ca005e49d7180e347: Status 404 returned error can't find the container with id 71b4855d5557e9dfe4182843bc43910a7a72ed9cd158119ca005e49d7180e347 Jul 02 14:16:38.554284 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.554261 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:16:38.555557 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.555534 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc468cb3a_4a0d_48db_8aae_80b251657d8a.slice/crio-0c7788142388d795464878ba46681b2a05082c7ebb8557777796a5a0e036dced WatchSource:0}: Error finding container 0c7788142388d795464878ba46681b2a05082c7ebb8557777796a5a0e036dced: Status 404 returned error can't find the container with id 0c7788142388d795464878ba46681b2a05082c7ebb8557777796a5a0e036dced Jul 02 14:16:38.558994 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.558962 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b41bd1d_1f33_40b9_95a0_2347fa038e15.slice/crio-80b698440ed052f78e659225f05b84a89c560dc6624cbd36ec58e7956f963afc WatchSource:0}: Error finding container 80b698440ed052f78e659225f05b84a89c560dc6624cbd36ec58e7956f963afc: Status 404 returned error can't find the container with id 80b698440ed052f78e659225f05b84a89c560dc6624cbd36ec58e7956f963afc Jul 02 14:16:38.560552 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.560518 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" Jul 02 14:16:38.566773 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.566085 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fsmwf" Jul 02 14:16:38.566773 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.566674 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5630e44_468e_4982_b742_d7b34aea53d2.slice/crio-a14390160f70b58bed60e466ee61b635e9aaf7746c0d058ed540a28c8ff598ff WatchSource:0}: Error finding container a14390160f70b58bed60e466ee61b635e9aaf7746c0d058ed540a28c8ff598ff: Status 404 returned error can't find the container with id a14390160f70b58bed60e466ee61b635e9aaf7746c0d058ed540a28c8ff598ff Jul 02 14:16:38.572916 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.572886 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15be550e_4122_4920_bc22_669d9b336248.slice/crio-34f4aa2e17436ba70dd2250b893a5d6b4c12e98745a12898b9a7d2990c0abfea WatchSource:0}: Error finding container 34f4aa2e17436ba70dd2250b893a5d6b4c12e98745a12898b9a7d2990c0abfea: Status 404 returned error can't find the container with id 34f4aa2e17436ba70dd2250b893a5d6b4c12e98745a12898b9a7d2990c0abfea Jul 02 14:16:38.578784 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:16:38.578759 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod545cf0b4_dd6f_489f_8f52_3c697db25831.slice/crio-425db86860de0886d69e4d0d0ec33d02534fb457d6a5dde55cf7352e794c7881 WatchSource:0}: Error finding container 425db86860de0886d69e4d0d0ec33d02534fb457d6a5dde55cf7352e794c7881: Status 404 returned error can't find the container with id 425db86860de0886d69e4d0d0ec33d02534fb457d6a5dde55cf7352e794c7881 Jul 02 14:16:38.831008 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.830979 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:38.831148 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.831028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:38.831148 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.831128 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:38.831276 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.831192 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:38.831276 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.831219 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:39.831200489 +0000 UTC m=+3.952442724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:38.831276 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.831248 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:39.831231851 +0000 UTC m=+3.952474086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:38.931622 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:38.931404 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:38.931782 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.931537 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:38.931782 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.931688 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:38.931782 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.931701 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:38.931782 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:38.931761 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:16:39.931741488 +0000 UTC m=+4.052983724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:39.263052 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.262857 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 14:11:38 +0000 UTC" deadline="2027-12-26 14:43:00.938211522 +0000 UTC" Jul 02 14:16:39.263052 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.262924 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13008h26m21.675292122s" Jul 02 14:16:39.289532 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.289377 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerStarted","Data":"0c7788142388d795464878ba46681b2a05082c7ebb8557777796a5a0e036dced"} Jul 02 14:16:39.291008 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.290977 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" event={"ID":"4911423d-9c72-42af-b0e0-73194c3f1c5e","Type":"ContainerStarted","Data":"22c3696392d0df08122f07d494fb66e67d0338d1ae1ebf30d603e6deea9aaa9b"} Jul 02 14:16:39.294500 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.294411 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e3f640dc2f40324fd180adbbfd0129f" containerID="d3257290f0955ebcf041ab9a484a5fa6946a8af72f3c1c81e30c36e5c891ad86" exitCode=0 Jul 02 14:16:39.294604 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.294503 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" event={"ID":"9e3f640dc2f40324fd180adbbfd0129f","Type":"ContainerDied","Data":"d3257290f0955ebcf041ab9a484a5fa6946a8af72f3c1c81e30c36e5c891ad86"} Jul 02 14:16:39.296016 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.295990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fsmwf" event={"ID":"545cf0b4-dd6f-489f-8f52-3c697db25831","Type":"ContainerStarted","Data":"425db86860de0886d69e4d0d0ec33d02534fb457d6a5dde55cf7352e794c7881"} Jul 02 14:16:39.297571 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.297490 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"a14390160f70b58bed60e466ee61b635e9aaf7746c0d058ed540a28c8ff598ff"} Jul 02 14:16:39.299751 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.299709 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6qkws" event={"ID":"9b41bd1d-1f33-40b9-95a0-2347fa038e15","Type":"ContainerStarted","Data":"80b698440ed052f78e659225f05b84a89c560dc6624cbd36ec58e7956f963afc"} Jul 02 14:16:39.301210 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.301186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9fsww" event={"ID":"a8ec2efe-70d4-4636-9f1a-914333629928","Type":"ContainerStarted","Data":"71b4855d5557e9dfe4182843bc43910a7a72ed9cd158119ca005e49d7180e347"} Jul 02 14:16:39.302460 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.302428 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kd2r7" event={"ID":"2bffbb99-79da-4149-8564-b3b77a67615f","Type":"ContainerStarted","Data":"1ea320829ff6d7ac59257650bd46f7e9bce68ad3d28a1fdd62664e095307fe8d"} Jul 02 14:16:39.304746 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.304723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qpqhq" event={"ID":"5fdfa295-ce5b-40f8-bb82-71d6ec26a879","Type":"ContainerStarted","Data":"edff550543193b696d04b1567101fc31c8298c42fe2fa19fe698e6102cfaf45e"} Jul 02 14:16:39.306052 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.306008 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" event={"ID":"15be550e-4122-4920-bc22-669d9b336248","Type":"ContainerStarted","Data":"34f4aa2e17436ba70dd2250b893a5d6b4c12e98745a12898b9a7d2990c0abfea"} Jul 02 14:16:39.839245 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.839206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:39.839430 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.839263 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:39.839430 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.839401 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:39.839561 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.839467 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:41.839444624 +0000 UTC m=+5.960686866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:39.839900 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.839880 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:39.839979 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.839935 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:41.839919878 +0000 UTC m=+5.961162097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:39.940985 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:39.940347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:39.940985 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.940559 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:39.940985 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.940578 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:39.940985 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.940592 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:39.940985 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:39.940654 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:16:41.940633837 +0000 UTC m=+6.061876074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:40.263452 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:40.263413 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 14:11:38 +0000 UTC" deadline="2028-02-13 08:10:58.641363196 +0000 UTC" Jul 02 14:16:40.263452 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:40.263453 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14177h54m18.37791536s" Jul 02 14:16:40.280303 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:40.280281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:40.280446 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:40.280409 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:40.280841 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:40.280813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:40.280950 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:40.280930 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:40.281184 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:40.281147 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:40.281265 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:40.281238 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:41.348572 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:41.348539 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" event={"ID":"9e3f640dc2f40324fd180adbbfd0129f","Type":"ContainerStarted","Data":"640d9f7d14fc77058b96686f6c5a8f0668fcec8eb42f8dba931aa4fd0d1ae1fe"} Jul 02 14:16:41.861041 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:41.860250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:41.861041 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:41.860301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:41.861041 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.860429 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:41.861041 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.860489 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:45.860471144 +0000 UTC m=+9.981713367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:41.861041 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.860854 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:41.861041 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.860901 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:45.860886683 +0000 UTC m=+9.982128906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:41.961084 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:41.960992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:41.961284 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.961197 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:41.961284 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.961228 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:41.961284 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.961241 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:41.961449 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:41.961301 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:16:45.961283239 +0000 UTC m=+10.082525460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:42.281049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:42.280540 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:42.281049 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:42.280672 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:42.281049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:42.280549 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:42.281049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:42.280720 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:42.281049 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:42.280778 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:42.281049 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:42.280860 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:44.279899 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:44.279821 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:44.279899 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:44.279854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:44.279899 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:44.279868 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:44.280428 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:44.279951 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:44.280428 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:44.280045 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:44.280428 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:44.280205 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:45.889897 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:45.889857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:45.890483 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:45.889912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:45.890483 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.889988 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:45.890483 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.890024 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:45.890483 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.890057 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:53.890041129 +0000 UTC m=+18.011283367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:45.890483 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.890079 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:16:53.890069691 +0000 UTC m=+18.011311914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:45.991027 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:45.990977 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:45.991224 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.991197 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:45.991286 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.991227 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:45.991286 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.991241 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:45.991394 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:45.991308 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:16:53.991288267 +0000 UTC m=+18.112530490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:46.281412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:46.280889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:46.281412 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:46.280997 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:46.281412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:46.281169 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:46.281412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:46.281199 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:46.281412 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:46.281288 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:46.281412 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:46.281367 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:48.279707 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:48.279679 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:48.280265 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:48.279714 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:48.280265 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:48.279713 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:48.280265 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:48.279800 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:48.280265 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:48.279925 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:48.280265 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:48.280028 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:50.280164 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:50.280125 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:50.280644 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:50.280179 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:50.280644 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:50.280125 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:50.280644 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:50.280271 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:50.280644 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:50.280356 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:50.280644 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:50.280411 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:52.280249 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:52.280202 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:52.280249 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:52.280239 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:52.280738 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:52.280203 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:52.280738 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:52.280331 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:52.280738 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:52.280398 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:52.280738 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:52.280491 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:53.953610 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:53.953573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:53.954092 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:53.953627 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:53.954092 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:53.953748 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:53.954092 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:53.953791 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:53.954092 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:53.953812 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:09.953797668 +0000 UTC m=+34.075039886 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:16:53.954092 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:53.953853 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:09.953835431 +0000 UTC m=+34.075077656 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:16:54.054166 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:54.054121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:54.054308 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.054268 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:16:54.054308 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.054287 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:16:54.054308 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.054301 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:54.054436 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.054366 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:17:10.054347319 +0000 UTC m=+34.175589536 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:16:54.280492 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:54.280420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:54.280659 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:54.280420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:54.280659 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:54.280420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:54.280659 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.280584 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:54.280821 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.280675 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:54.280821 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:54.280776 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:56.280594 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:56.280555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:56.281089 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:56.280675 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:56.281089 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:56.280677 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:56.281089 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:56.280762 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:56.281089 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:56.280769 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:56.281089 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:56.280884 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:57.390234 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:57.390168 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" event={"ID":"6477d183aafb4ad9f6bb122000884dcf","Type":"ContainerStarted","Data":"bac994675dc22b973cf97e7dd2a11241c0525575b723da2cd2de33355abbb73c"} Jul 02 14:16:57.413989 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:57.413925 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-249.ec2.internal" podStartSLOduration=20.413905325 podStartE2EDuration="20.413905325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:41.366620973 +0000 UTC m=+5.487863211" watchObservedRunningTime="2026-07-02 14:16:57.413905325 +0000 UTC m=+21.535147569" Jul 02 14:16:57.415681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:57.414282 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-249.ec2.internal" podStartSLOduration=20.414270652 podStartE2EDuration="20.414270652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:57.413589074 +0000 UTC m=+21.534831316" watchObservedRunningTime="2026-07-02 14:16:57.414270652 +0000 UTC m=+21.535512895" Jul 02 14:16:58.280393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.280186 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:16:58.280554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.280193 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:16:58.280554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.280241 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:16:58.280554 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:58.280496 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:16:58.280712 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:58.280593 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:16:58.280712 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:16:58.280674 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:16:58.396766 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.396556 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fsmwf" event={"ID":"545cf0b4-dd6f-489f-8f52-3c697db25831","Type":"ContainerStarted","Data":"d4d4cbfb15168457b0e2f486be1a73024795790b4b1ded5918efc87359d1509f"} Jul 02 14:16:58.398054 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.398027 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"e73024b649a13facdd83b4dd61f9f05c1da535d9412a44e42d9f8af3aa27943e"} Jul 02 14:16:58.399289 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.399263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6qkws" event={"ID":"9b41bd1d-1f33-40b9-95a0-2347fa038e15","Type":"ContainerStarted","Data":"612a87ba5841bbb96e498789c21f31f45a0e189919cbf002fe5fe0dc3eeeeaca"} Jul 02 14:16:58.400515 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.400490 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9fsww" event={"ID":"a8ec2efe-70d4-4636-9f1a-914333629928","Type":"ContainerStarted","Data":"b622b6864983504d1942e9daca84a1dc003077cd36b289d8be68d323281d49f7"} Jul 02 14:16:58.401873 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.401849 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kd2r7" event={"ID":"2bffbb99-79da-4149-8564-b3b77a67615f","Type":"ContainerStarted","Data":"f87f0b2bfc7c1616780fe6022ebedbfec60d269f50231ef31132ea0fc5a52e8e"} Jul 02 14:16:58.403343 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.403305 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" event={"ID":"15be550e-4122-4920-bc22-669d9b336248","Type":"ContainerStarted","Data":"13fc43b930fe25aa3b902eb16d12b7c57d6dceeee33fab8e2cfb671280024816"} Jul 02 14:16:58.404662 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.404639 2581 generic.go:356] "Generic (PLEG): container finished" podID="c468cb3a-4a0d-48db-8aae-80b251657d8a" containerID="349c0892a1ddcac22afa16cdc091e668dd5b243beb280197b95e6e70e2f3da5b" exitCode=0 Jul 02 14:16:58.404744 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.404713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerDied","Data":"349c0892a1ddcac22afa16cdc091e668dd5b243beb280197b95e6e70e2f3da5b"} Jul 02 14:16:58.406133 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.406111 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" event={"ID":"4911423d-9c72-42af-b0e0-73194c3f1c5e","Type":"ContainerStarted","Data":"468b12286b89f0f27b9562bf78fd368762393c5812992929a47a87480c7864fe"} Jul 02 14:16:58.412858 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.412821 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fsmwf" podStartSLOduration=3.770100067 podStartE2EDuration="22.412809612s" totalImagesPullingTime="18.642709545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:58.412618569 +0000 UTC m=+22.533860823" watchObservedRunningTime="2026-07-02 14:16:58.412809612 +0000 UTC m=+22.534051852" Jul 02 14:16:58.432348 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.432304 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9fsww" podStartSLOduration=3.702243561 podStartE2EDuration="22.432287145s" totalImagesPullingTime="18.730043584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:58.431875755 +0000 UTC m=+22.553117995" watchObservedRunningTime="2026-07-02 14:16:58.432287145 +0000 UTC m=+22.553529386" Jul 02 14:16:58.464636 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.464594 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c5vtl" podStartSLOduration=3.814220048 podStartE2EDuration="22.46457796s" totalImagesPullingTime="18.650357912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:58.448758831 +0000 UTC m=+22.570001071" watchObservedRunningTime="2026-07-02 14:16:58.46457796 +0000 UTC m=+22.585820200" Jul 02 14:16:58.490547 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.490499 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kd2r7" podStartSLOduration=3.82086659 podStartE2EDuration="22.490485304s" totalImagesPullingTime="18.669618714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:58.464435579 +0000 UTC m=+22.585677820" watchObservedRunningTime="2026-07-02 14:16:58.490485304 +0000 UTC m=+22.611727543" Jul 02 14:16:58.506837 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.506791 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6qkws" podStartSLOduration=3.847273146 podStartE2EDuration="22.506779563s" totalImagesPullingTime="18.659506417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:58.506261671 +0000 UTC m=+22.627503912" watchObservedRunningTime="2026-07-02 14:16:58.506779563 +0000 UTC m=+22.628021802" Jul 02 14:16:58.761912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:58.761816 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 14:16:59.283015 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.282864 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T14:16:58.761834233Z","UUID":"483d257a-571e-479f-bf30-45753675487f","Handler":null,"Name":"","Endpoint":""} Jul 02 14:16:59.284637 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.284605 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 14:16:59.284757 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.284651 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 14:16:59.412053 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.412014 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"a565ae5b2a488e57a0813302569da663047137e1a05e73cc619d04d14e65cbd8"} Jul 02 14:16:59.412559 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.412148 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"d1954afe76bc19f51f298fd9514d5e6aa7f9b0aa047c67184e41c4d5e0a9e327"} Jul 02 14:16:59.412559 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.412226 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"07e86b395ae0bfa64176f63b8c9ba3f6eb4465935a3d59ea9e686f3951bd4e33"} Jul 02 14:16:59.412559 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.412267 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"7bf3b3a1dd092bc37551dec4335a29a8d1877f3404742b799ea9a5c87a137695"} Jul 02 14:16:59.412559 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.412313 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"3635ed5791054d4aca64cd335b554b676abb21f6021ea4f368194be68fc0ddd9"} Jul 02 14:16:59.413680 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.413647 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qpqhq" event={"ID":"5fdfa295-ce5b-40f8-bb82-71d6ec26a879","Type":"ContainerStarted","Data":"fd8fe9b843b19484d7c206831faef7c779ae5f680c2ed7f886f36d9bf775c97e"} Jul 02 14:16:59.415526 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.415501 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" event={"ID":"4911423d-9c72-42af-b0e0-73194c3f1c5e","Type":"ContainerStarted","Data":"4376a1300ebb0dadc721e2dcbbdb9d26b86bb091962a88ced1138db83318b339"} Jul 02 14:16:59.430178 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:16:59.430123 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qpqhq" podStartSLOduration=4.7320013979999995 podStartE2EDuration="23.430087683s" totalImagesPullingTime="18.698086285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:16:59.430041403 +0000 UTC m=+23.551283644" watchObservedRunningTime="2026-07-02 14:16:59.430087683 +0000 UTC m=+23.551329926" Jul 02 14:17:00.004123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:00.004089 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:17:00.004775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:00.004749 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:17:00.280018 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:00.279923 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:00.280018 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:00.279951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:00.280282 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:00.279923 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:00.280282 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:00.280055 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:17:00.280282 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:00.280137 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:17:00.280282 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:00.280239 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:17:01.422956 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:01.422701 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" event={"ID":"4911423d-9c72-42af-b0e0-73194c3f1c5e","Type":"ContainerStarted","Data":"0702b0c65822fa52587cbc4cc11bfb2983a99d36679646709e1c04ec737e9ca2"} Jul 02 14:17:01.423429 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:01.422745 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 14:17:01.448371 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:01.448313 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8tw4r" podStartSLOduration=8.536978235 podStartE2EDuration="25.448293983s" totalImagesPullingTime="16.911315748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:01.448010804 +0000 UTC m=+25.569253046" watchObservedRunningTime="2026-07-02 14:17:01.448293983 +0000 UTC m=+25.569536222" Jul 02 14:17:02.283305 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.283276 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:02.283465 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.283360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:02.283465 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:02.283363 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:17:02.283465 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.283386 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:02.283632 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:02.283465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:17:02.283632 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:02.283531 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:17:02.428750 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.428715 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"2de26c867a9a7048255d52a61d58b37e5d478d019a7a76af907e63a61c4c4fbf"} Jul 02 14:17:02.496376 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.496342 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:17:02.496564 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.496495 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 02 14:17:02.496979 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:02.496961 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6qkws" Jul 02 14:17:03.432946 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:03.432914 2581 generic.go:356] "Generic (PLEG): container finished" podID="c468cb3a-4a0d-48db-8aae-80b251657d8a" containerID="f1e0669c50c3bcab592203eb994ec8e331af3f8f115ba906a6f805c221647f91" exitCode=0 Jul 02 14:17:03.433498 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:03.432973 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerDied","Data":"f1e0669c50c3bcab592203eb994ec8e331af3f8f115ba906a6f805c221647f91"} Jul 02 14:17:04.280215 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:04.280176 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:04.280215 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:04.280214 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:04.280443 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:04.280180 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:04.280443 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:04.280300 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:17:04.280443 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:04.280351 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:17:04.280443 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:04.280432 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:17:05.440218 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.440006 2581 generic.go:356] "Generic (PLEG): container finished" podID="c468cb3a-4a0d-48db-8aae-80b251657d8a" containerID="c85dcf3ee8dd35a00cea7619a5e845fb918bb77b50591452d97777ec4a357d05" exitCode=0 Jul 02 14:17:05.440700 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.440065 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerDied","Data":"c85dcf3ee8dd35a00cea7619a5e845fb918bb77b50591452d97777ec4a357d05"} Jul 02 14:17:05.444109 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.444084 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" event={"ID":"c5630e44-468e-4982-b742-d7b34aea53d2","Type":"ContainerStarted","Data":"7010a3cf1d041c1827d4a18a4c940cdadc9a5b287e5ae601dc117bbc5a516b9c"} Jul 02 14:17:05.444420 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.444398 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:17:05.444511 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.444474 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:17:05.444565 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.444527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:17:05.459736 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.459702 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:17:05.459838 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.459818 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:17:05.498259 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:05.498212 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" podStartSLOduration=10.018403914 podStartE2EDuration="29.498196114s" totalImagesPullingTime="19.4797922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:05.497620309 +0000 UTC m=+29.618862548" watchObservedRunningTime="2026-07-02 14:17:05.498196114 +0000 UTC m=+29.619438356" Jul 02 14:17:06.280622 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.280585 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:06.280766 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:06.280687 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:17:06.280766 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.280707 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:06.280766 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.280763 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:06.280923 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:06.280855 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:17:06.281018 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:06.280987 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:17:06.448882 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.448844 2581 generic.go:356] "Generic (PLEG): container finished" podID="c468cb3a-4a0d-48db-8aae-80b251657d8a" containerID="671f65e789a1f33ab1e1418a6098f0ff5cd2d525f89b9e014212320a7664de46" exitCode=0 Jul 02 14:17:06.449313 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.448930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerDied","Data":"671f65e789a1f33ab1e1418a6098f0ff5cd2d525f89b9e014212320a7664de46"} Jul 02 14:17:06.814860 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.814674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dbr7t"] Jul 02 14:17:06.815003 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.814947 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:06.815055 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:06.815037 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:17:06.819137 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.819107 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4mhz9"] Jul 02 14:17:06.819296 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.819267 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:06.819420 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:06.819395 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:17:06.819939 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.819916 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8vhdd"] Jul 02 14:17:06.820043 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:06.820012 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:06.820123 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:06.820103 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:17:08.282917 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:08.282888 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:08.282917 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:08.282914 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:08.283983 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:08.282996 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:08.283983 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:08.283013 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mhz9" podUID="7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3" Jul 02 14:17:08.283983 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:08.283065 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dbr7t" podUID="cdc7ec1e-5d10-4369-b9a3-0e0f77b38757" Jul 02 14:17:08.283983 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:08.283172 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8vhdd" podUID="43db889f-6d33-4460-a927-c974412e98dd" Jul 02 14:17:09.967035 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:09.966998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:09.967441 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:09.967052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:09.967441 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:09.967222 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 14:17:09.967441 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:09.967290 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret podName:cdc7ec1e-5d10-4369-b9a3-0e0f77b38757 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:41.96727384 +0000 UTC m=+66.088516062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret") pod "global-pull-secret-syncer-dbr7t" (UID: "cdc7ec1e-5d10-4369-b9a3-0e0f77b38757") : object "kube-system"/"original-pull-secret" not registered Jul 02 14:17:09.967441 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:09.967228 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:17:09.967441 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:09.967353 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:41.96734007 +0000 UTC m=+66.088582288 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 14:17:10.067902 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.067866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:10.068071 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.068035 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 14:17:10.068071 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.068051 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 14:17:10.068071 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.068060 2581 projected.go:196] Error preparing data for projected volume kube-api-access-xhzsq for pod openshift-network-diagnostics/network-check-target-8vhdd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:17:10.068267 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.068103 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq podName:43db889f-6d33-4460-a927-c974412e98dd nodeName:}" failed. No retries permitted until 2026-07-02 14:17:42.068090798 +0000 UTC m=+66.189333016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xhzsq" (UniqueName: "kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq") pod "network-check-target-8vhdd" (UID: "43db889f-6d33-4460-a927-c974412e98dd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 14:17:10.113317 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.113280 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-249.ec2.internal" event="NodeReady" Jul 02 14:17:10.113472 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.113434 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 14:17:10.162757 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.162729 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tgp6n"] Jul 02 14:17:10.165465 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.165443 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.166053 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.166023 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c9p6q"] Jul 02 14:17:10.167815 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.167795 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.168105 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.168042 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 14:17:10.168229 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.168049 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 14:17:10.168287 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.168226 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mttc5\"" Jul 02 14:17:10.170346 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.170324 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 14:17:10.170346 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.170339 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 14:17:10.171055 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.170878 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 14:17:10.171055 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.171045 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z4mlq\"" Jul 02 14:17:10.178864 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.178408 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tgp6n"] Jul 02 14:17:10.179246 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.179227 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c9p6q"] Jul 02 14:17:10.269564 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.269489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/978d29b1-a9d0-44ee-95d2-11572d394c92-config-volume\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.269564 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.269526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzvjq\" (UniqueName: \"kubernetes.io/projected/978d29b1-a9d0-44ee-95d2-11572d394c92-kube-api-access-mzvjq\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.269564 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.269558 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a1b3fcc-b59c-480d-82eb-87a29f8cb07a-cert\") pod \"ingress-canary-c9p6q\" (UID: \"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a\") " pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.269802 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.269641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.269802 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.269683 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978d29b1-a9d0-44ee-95d2-11572d394c92-tmp-dir\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.269802 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.269705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqml\" (UniqueName: \"kubernetes.io/projected/9a1b3fcc-b59c-480d-82eb-87a29f8cb07a-kube-api-access-dxqml\") pod \"ingress-canary-c9p6q\" (UID: \"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a\") " pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.280570 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.280543 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:10.280696 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.280571 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:10.280696 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.280587 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:10.283932 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.283853 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 14:17:10.284062 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.284024 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8mqz5\"" Jul 02 14:17:10.284207 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.284183 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 14:17:10.284333 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.284241 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q6bk5\"" Jul 02 14:17:10.284333 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.284259 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 14:17:10.284333 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.284313 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 14:17:10.370962 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.370928 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978d29b1-a9d0-44ee-95d2-11572d394c92-tmp-dir\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.371123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.370969 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqml\" (UniqueName: \"kubernetes.io/projected/9a1b3fcc-b59c-480d-82eb-87a29f8cb07a-kube-api-access-dxqml\") pod \"ingress-canary-c9p6q\" (UID: \"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a\") " pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.371123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.371030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/978d29b1-a9d0-44ee-95d2-11572d394c92-config-volume\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.371123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.371054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzvjq\" (UniqueName: \"kubernetes.io/projected/978d29b1-a9d0-44ee-95d2-11572d394c92-kube-api-access-mzvjq\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.371123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.371080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a1b3fcc-b59c-480d-82eb-87a29f8cb07a-cert\") pod \"ingress-canary-c9p6q\" (UID: \"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a\") " pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.371341 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.371142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.371341 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.371292 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:10.371341 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.371311 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978d29b1-a9d0-44ee-95d2-11572d394c92-tmp-dir\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.371475 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.371362 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:10.871342601 +0000 UTC m=+34.992584822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:10.371587 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.371569 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/978d29b1-a9d0-44ee-95d2-11572d394c92-config-volume\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.375307 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.375286 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a1b3fcc-b59c-480d-82eb-87a29f8cb07a-cert\") pod \"ingress-canary-c9p6q\" (UID: \"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a\") " pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.381853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.381829 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzvjq\" (UniqueName: \"kubernetes.io/projected/978d29b1-a9d0-44ee-95d2-11572d394c92-kube-api-access-mzvjq\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.382658 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.382641 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqml\" (UniqueName: \"kubernetes.io/projected/9a1b3fcc-b59c-480d-82eb-87a29f8cb07a-kube-api-access-dxqml\") pod \"ingress-canary-c9p6q\" (UID: \"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a\") " pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.484963 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.484928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c9p6q" Jul 02 14:17:10.874924 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:10.874883 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:10.875100 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.875035 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:10.875143 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:10.875106 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:11.875090435 +0000 UTC m=+35.996332656 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:11.487592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:11.487396 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c9p6q"] Jul 02 14:17:11.492999 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:11.492969 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1b3fcc_b59c_480d_82eb_87a29f8cb07a.slice/crio-a426dabb1532ca1c8a39cccbdadbfde65fc114cd359d35c8208fc9820e6d3ea8 WatchSource:0}: Error finding container a426dabb1532ca1c8a39cccbdadbfde65fc114cd359d35c8208fc9820e6d3ea8: Status 404 returned error can't find the container with id a426dabb1532ca1c8a39cccbdadbfde65fc114cd359d35c8208fc9820e6d3ea8 Jul 02 14:17:11.881525 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:11.881492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:11.881678 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:11.881658 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:11.881735 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:11.881726 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:13.881710692 +0000 UTC m=+38.002952909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:12.467925 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:12.467886 2581 generic.go:356] "Generic (PLEG): container finished" podID="c468cb3a-4a0d-48db-8aae-80b251657d8a" containerID="dc87c2fcd759e705017898f2d28bf6cb46942918785fff159af4134cff7415b9" exitCode=0 Jul 02 14:17:12.468103 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:12.467980 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerDied","Data":"dc87c2fcd759e705017898f2d28bf6cb46942918785fff159af4134cff7415b9"} Jul 02 14:17:12.469333 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:12.469282 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c9p6q" event={"ID":"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a","Type":"ContainerStarted","Data":"a426dabb1532ca1c8a39cccbdadbfde65fc114cd359d35c8208fc9820e6d3ea8"} Jul 02 14:17:13.475270 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:13.475231 2581 generic.go:356] "Generic (PLEG): container finished" podID="c468cb3a-4a0d-48db-8aae-80b251657d8a" containerID="9f1d7ec0fc088ac881cd237c6c223f49572720f00e54bba9aef2599753c4a026" exitCode=0 Jul 02 14:17:13.475756 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:13.475272 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerDied","Data":"9f1d7ec0fc088ac881cd237c6c223f49572720f00e54bba9aef2599753c4a026"} Jul 02 14:17:13.897177 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:13.897130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:13.897301 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:13.897276 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:13.897349 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:13.897338 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:17.897321026 +0000 UTC m=+42.018563260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:14.386094 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.385926 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:14.472571 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.472537 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:14.480408 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.480383 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" event={"ID":"c468cb3a-4a0d-48db-8aae-80b251657d8a","Type":"ContainerStarted","Data":"38c7dc04d6b9a1df3c56b6045012c36c6ffdce06ed38b66a9d59576a6f983779"} Jul 02 14:17:14.481744 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.481723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c9p6q" event={"ID":"9a1b3fcc-b59c-480d-82eb-87a29f8cb07a","Type":"ContainerStarted","Data":"f295a0aa9d0bbd8e329589bfd5671d7c76c80818bee0be841e6fbab2aa0f6253"} Jul 02 14:17:14.509911 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.509863 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2rxvc" podStartSLOduration=8.773986608 podStartE2EDuration="38.509849001s" totalImagesPullingTime="29.735862393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:14.508348034 +0000 UTC m=+38.629590272" watchObservedRunningTime="2026-07-02 14:17:14.509849001 +0000 UTC m=+38.631091240" Jul 02 14:17:14.526143 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.526093 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c9p6q" podStartSLOduration=2.007151878 podStartE2EDuration="4.52607018s" totalImagesPullingTime="2.518918302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:14.52531929 +0000 UTC m=+38.646561530" watchObservedRunningTime="2026-07-02 14:17:14.52607018 +0000 UTC m=+38.647312420" Jul 02 14:17:14.642276 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.642208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:14.746632 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:14.746600 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:15.305076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:15.305042 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:15.379953 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:15.379920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:16.168325 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:16.168297 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:16.250450 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:16.250419 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:17.921416 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:17.921379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:17.921870 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:17.921533 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:17.921870 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:17.921596 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:25.92157745 +0000 UTC m=+50.042819672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:18.471739 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:18.471708 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:18.529897 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:18.529864 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:19.553910 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:19.553876 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:19.612732 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:19.612704 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:25.977556 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:25.977515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:25.977960 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:25.977629 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:25.977960 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:25.977692 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:17:41.97767463 +0000 UTC m=+66.098916850 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:32.267962 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:32.267927 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:32.369946 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:32.369919 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:36.130881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:36.130844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:36.190093 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:36.190056 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:37.140189 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.140007 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp"] Jul 02 14:17:37.144183 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.144139 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.144524 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.144502 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-fsmx9"] Jul 02 14:17:37.147290 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.147270 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.147697 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.147681 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-txt8h\"" Jul 02 14:17:37.147945 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.147921 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:17:37.148995 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.148975 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 02 14:17:37.149246 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.149228 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 02 14:17:37.149902 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.149885 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-555b667bcf-7v5d7"] Jul 02 14:17:37.151408 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.151392 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 14:17:37.151408 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.151405 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 14:17:37.151776 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.151758 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-q6zxl"] Jul 02 14:17:37.151853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.151833 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.152472 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.152451 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 02 14:17:37.152557 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.152472 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 02 14:17:37.152621 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.152601 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wcndc\"" Jul 02 14:17:37.153875 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.153855 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.153971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.153935 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp"] Jul 02 14:17:37.154911 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.154896 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 14:17:37.155001 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.154925 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hxdbc\"" Jul 02 14:17:37.155134 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.155118 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 14:17:37.155776 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.155758 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 14:17:37.159758 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.159728 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 02 14:17:37.159758 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.159735 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:17:37.159898 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.159760 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 02 14:17:37.160331 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.160312 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 02 14:17:37.160433 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.160370 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-26mx2\"" Jul 02 14:17:37.160877 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.160858 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-fsmx9"] Jul 02 14:17:37.167970 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.167951 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 02 14:17:37.168046 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.168006 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 14:17:37.169810 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.169794 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555b667bcf-7v5d7"] Jul 02 14:17:37.169901 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.169887 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 02 14:17:37.185681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.185661 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-q6zxl"] Jul 02 14:17:37.235719 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.235688 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk"] Jul 02 14:17:37.240398 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.240373 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" Jul 02 14:17:37.242975 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.242956 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 02 14:17:37.243066 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.242982 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-c57wf\"" Jul 02 14:17:37.243127 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.243089 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:17:37.247383 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.247355 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk"] Jul 02 14:17:37.252310 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252293 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-serving-cert\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.252409 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-installation-pull-secrets\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252409 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-trusted-ca\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252409 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q8vq\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-kube-api-access-9q8vq\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252519 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252457 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:37.252519 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.252585 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252523 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.252585 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cr7f\" (UniqueName: \"kubernetes.io/projected/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-kube-api-access-8cr7f\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.252585 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcdqb\" (UniqueName: \"kubernetes.io/projected/f98de5fa-8b99-409c-b217-eff19b9b536e-kube-api-access-rcdqb\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.252676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-snapshots\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.252676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252633 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-certificates\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f98de5fa-8b99-409c-b217-eff19b9b536e-serving-cert\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.252676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98de5fa-8b99-409c-b217-eff19b9b536e-trusted-ca\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.252818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252704 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-ca-trust-extracted\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252724 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mqpt\" (UniqueName: \"kubernetes.io/projected/0476e7a3-b391-499b-877d-e0402cacc30c-kube-api-access-9mqpt\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.252818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-tmp\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.252818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-image-registry-private-configuration\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-bound-sa-token\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.252818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252790 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f98de5fa-8b99-409c-b217-eff19b9b536e-config\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.252981 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.252842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-service-ca-bundle\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.324087 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.324060 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:37.353213 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-ca-trust-extracted\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353320 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-bound-sa-token\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353320 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-trusted-ca\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353320 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q8vq\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-kube-api-access-9q8vq\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353416 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353330 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-snapshots\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.353416 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353352 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.353416 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.353416 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353393 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcdqb\" (UniqueName: \"kubernetes.io/projected/f98de5fa-8b99-409c-b217-eff19b9b536e-kube-api-access-rcdqb\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-certificates\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-tmp\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-image-registry-private-configuration\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353526 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f98de5fa-8b99-409c-b217-eff19b9b536e-config\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.353523 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mqpt\" (UniqueName: \"kubernetes.io/projected/0476e7a3-b391-499b-877d-e0402cacc30c-kube-api-access-9mqpt\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.353592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-service-ca-bundle\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.353947 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.353634 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls podName:0476e7a3-b391-499b-877d-e0402cacc30c nodeName:}" failed. No retries permitted until 2026-07-02 14:17:37.853596812 +0000 UTC m=+61.974839047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-hc7dp" (UID: "0476e7a3-b391-499b-877d-e0402cacc30c") : secret "samples-operator-tls" not found Jul 02 14:17:37.353947 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-serving-cert\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.353947 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g955v\" (UniqueName: \"kubernetes.io/projected/32cd5102-1e48-45fe-b380-10c49eeaa8cf-kube-api-access-g955v\") pod \"volume-data-source-validator-69d9f679db-ck2wk\" (UID: \"32cd5102-1e48-45fe-b380-10c49eeaa8cf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" Jul 02 14:17:37.353947 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.353783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-installation-pull-secrets\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354004 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-snapshots\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.354064 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.354077 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555b667bcf-7v5d7: secret "image-registry-tls" not found Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f98de5fa-8b99-409c-b217-eff19b9b536e-serving-cert\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.354131 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls podName:8df61a8c-8b5d-495f-ab5c-0968f26ed55e nodeName:}" failed. No retries permitted until 2026-07-02 14:17:37.854116204 +0000 UTC m=+61.975358421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls") pod "image-registry-555b667bcf-7v5d7" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e") : secret "image-registry-tls" not found Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354130 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-ca-trust-extracted\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.354202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cr7f\" (UniqueName: \"kubernetes.io/projected/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-kube-api-access-8cr7f\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.354712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98de5fa-8b99-409c-b217-eff19b9b536e-trusted-ca\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.354712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354337 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-trusted-ca\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.354712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354346 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.354712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354452 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f98de5fa-8b99-409c-b217-eff19b9b536e-config\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.354712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354489 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-service-ca-bundle\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.354989 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.354845 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-tmp\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.355059 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.355038 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-certificates\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.355105 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.355064 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f98de5fa-8b99-409c-b217-eff19b9b536e-trusted-ca\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.357868 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.357839 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-serving-cert\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.357950 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.357882 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-installation-pull-secrets\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.357950 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.357920 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f98de5fa-8b99-409c-b217-eff19b9b536e-serving-cert\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.358018 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.357946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-image-registry-private-configuration\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.362455 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.362402 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-bound-sa-token\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.362801 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.362782 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcdqb\" (UniqueName: \"kubernetes.io/projected/f98de5fa-8b99-409c-b217-eff19b9b536e-kube-api-access-rcdqb\") pod \"console-operator-7fdbdf9c99-q6zxl\" (UID: \"f98de5fa-8b99-409c-b217-eff19b9b536e\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.362963 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.362939 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q8vq\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-kube-api-access-9q8vq\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.364667 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.364650 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cr7f\" (UniqueName: \"kubernetes.io/projected/3e2045a4-0e5b-483b-9f85-43d1ab8b8d33-kube-api-access-8cr7f\") pod \"insights-operator-5f67b55555-fsmx9\" (UID: \"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33\") " pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.365049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.365027 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mqpt\" (UniqueName: \"kubernetes.io/projected/0476e7a3-b391-499b-877d-e0402cacc30c-kube-api-access-9mqpt\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.455178 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.455078 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g955v\" (UniqueName: \"kubernetes.io/projected/32cd5102-1e48-45fe-b380-10c49eeaa8cf-kube-api-access-g955v\") pod \"volume-data-source-validator-69d9f679db-ck2wk\" (UID: \"32cd5102-1e48-45fe-b380-10c49eeaa8cf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" Jul 02 14:17:37.460875 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.460855 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m7grd" Jul 02 14:17:37.462435 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.462421 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" Jul 02 14:17:37.464806 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.464777 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g955v\" (UniqueName: \"kubernetes.io/projected/32cd5102-1e48-45fe-b380-10c49eeaa8cf-kube-api-access-g955v\") pod \"volume-data-source-validator-69d9f679db-ck2wk\" (UID: \"32cd5102-1e48-45fe-b380-10c49eeaa8cf\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" Jul 02 14:17:37.475352 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.475333 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:37.551746 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.551425 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" Jul 02 14:17:37.591509 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.591361 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:37.628495 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.628468 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-q6zxl"] Jul 02 14:17:37.631895 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:37.631867 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf98de5fa_8b99_409c_b217_eff19b9b536e.slice/crio-6db5e87914a484ccd5822c03c4dece84dc70a9636ba67abc95760de4a67fd9ed WatchSource:0}: Error finding container 6db5e87914a484ccd5822c03c4dece84dc70a9636ba67abc95760de4a67fd9ed: Status 404 returned error can't find the container with id 6db5e87914a484ccd5822c03c4dece84dc70a9636ba67abc95760de4a67fd9ed Jul 02 14:17:37.645823 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.645784 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-fsmx9"] Jul 02 14:17:37.649188 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:37.649148 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2045a4_0e5b_483b_9f85_43d1ab8b8d33.slice/crio-29ed2fe12212598ec03939dfd065ab4d1da8aa0823736dfdf43ae859c9b7f40e WatchSource:0}: Error finding container 29ed2fe12212598ec03939dfd065ab4d1da8aa0823736dfdf43ae859c9b7f40e: Status 404 returned error can't find the container with id 29ed2fe12212598ec03939dfd065ab4d1da8aa0823736dfdf43ae859c9b7f40e Jul 02 14:17:37.657687 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.657660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:37.699055 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.699025 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk"] Jul 02 14:17:37.701724 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:37.701688 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32cd5102_1e48_45fe_b380_10c49eeaa8cf.slice/crio-30a795ab448aa0a9f2ed184eb42ba35a1a8c148f5d38232e9a83b2dc9e8023c3 WatchSource:0}: Error finding container 30a795ab448aa0a9f2ed184eb42ba35a1a8c148f5d38232e9a83b2dc9e8023c3: Status 404 returned error can't find the container with id 30a795ab448aa0a9f2ed184eb42ba35a1a8c148f5d38232e9a83b2dc9e8023c3 Jul 02 14:17:37.859004 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.858976 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:37.859140 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:37.859013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:37.859140 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.859115 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:17:37.859140 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.859119 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 14:17:37.859277 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.859199 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls podName:0476e7a3-b391-499b-877d-e0402cacc30c nodeName:}" failed. No retries permitted until 2026-07-02 14:17:38.8591844 +0000 UTC m=+62.980426618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-hc7dp" (UID: "0476e7a3-b391-499b-877d-e0402cacc30c") : secret "samples-operator-tls" not found Jul 02 14:17:37.859277 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.859126 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555b667bcf-7v5d7: secret "image-registry-tls" not found Jul 02 14:17:37.859277 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:37.859249 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls podName:8df61a8c-8b5d-495f-ab5c-0968f26ed55e nodeName:}" failed. No retries permitted until 2026-07-02 14:17:38.859235879 +0000 UTC m=+62.980478096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls") pod "image-registry-555b667bcf-7v5d7" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e") : secret "image-registry-tls" not found Jul 02 14:17:38.008394 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.008318 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn"] Jul 02 14:17:38.011089 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.011073 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.013792 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.013770 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 02 14:17:38.013886 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.013769 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-947ts\"" Jul 02 14:17:38.014049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.014035 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 02 14:17:38.014113 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.014036 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 02 14:17:38.014180 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.014063 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 02 14:17:38.020709 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.020688 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn"] Jul 02 14:17:38.161298 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.161267 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.161688 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.161378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.161688 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.161458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqnw\" (UniqueName: \"kubernetes.io/projected/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-kube-api-access-trqnw\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.262705 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.262629 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.262859 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.262730 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqnw\" (UniqueName: \"kubernetes.io/projected/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-kube-api-access-trqnw\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.262859 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.262760 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.263355 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.263323 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.265199 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.265178 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.273253 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.273229 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqnw\" (UniqueName: \"kubernetes.io/projected/87a7e4f0-15dd-4ee3-bf66-9a0033fa496c-kube-api-access-trqnw\") pod \"kube-storage-version-migrator-operator-79964ff6f9-4dnzn\" (UID: \"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.320600 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.320521 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" Jul 02 14:17:38.470782 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.470299 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn"] Jul 02 14:17:38.473742 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:38.473708 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a7e4f0_15dd_4ee3_bf66_9a0033fa496c.slice/crio-375f3e043367f74dabbf7be97177ddc78d5ca4e653baed98010a64da848b5daf WatchSource:0}: Error finding container 375f3e043367f74dabbf7be97177ddc78d5ca4e653baed98010a64da848b5daf: Status 404 returned error can't find the container with id 375f3e043367f74dabbf7be97177ddc78d5ca4e653baed98010a64da848b5daf Jul 02 14:17:38.548039 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.547948 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" event={"ID":"32cd5102-1e48-45fe-b380-10c49eeaa8cf","Type":"ContainerStarted","Data":"30a795ab448aa0a9f2ed184eb42ba35a1a8c148f5d38232e9a83b2dc9e8023c3"} Jul 02 14:17:38.549435 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.549410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" event={"ID":"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c","Type":"ContainerStarted","Data":"375f3e043367f74dabbf7be97177ddc78d5ca4e653baed98010a64da848b5daf"} Jul 02 14:17:38.551296 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.551239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" event={"ID":"f98de5fa-8b99-409c-b217-eff19b9b536e","Type":"ContainerStarted","Data":"6db5e87914a484ccd5822c03c4dece84dc70a9636ba67abc95760de4a67fd9ed"} Jul 02 14:17:38.553058 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.553033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" event={"ID":"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33","Type":"ContainerStarted","Data":"29ed2fe12212598ec03939dfd065ab4d1da8aa0823736dfdf43ae859c9b7f40e"} Jul 02 14:17:38.868646 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.868409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:38.868646 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:38.868466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:38.868861 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:38.868683 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 14:17:38.868861 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:38.868734 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:17:38.868861 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:38.868755 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555b667bcf-7v5d7: secret "image-registry-tls" not found Jul 02 14:17:38.868861 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:38.868755 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls podName:0476e7a3-b391-499b-877d-e0402cacc30c nodeName:}" failed. No retries permitted until 2026-07-02 14:17:40.86873447 +0000 UTC m=+64.989976691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-hc7dp" (UID: "0476e7a3-b391-499b-877d-e0402cacc30c") : secret "samples-operator-tls" not found Jul 02 14:17:38.868861 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:38.868816 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls podName:8df61a8c-8b5d-495f-ab5c-0968f26ed55e nodeName:}" failed. No retries permitted until 2026-07-02 14:17:40.868799206 +0000 UTC m=+64.990041430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls") pod "image-registry-555b667bcf-7v5d7" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e") : secret "image-registry-tls" not found Jul 02 14:17:39.557738 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:39.557698 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" event={"ID":"32cd5102-1e48-45fe-b380-10c49eeaa8cf","Type":"ContainerStarted","Data":"d69766b15b31740a0b475e16d242fac9a8f7b43bf537b8bbc82f2756600a6fed"} Jul 02 14:17:39.574507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:39.574449 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-ck2wk" podStartSLOduration=1.154772745 podStartE2EDuration="2.574428204s" totalImagesPullingTime="1.419655459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:39.573604878 +0000 UTC m=+63.694847119" watchObservedRunningTime="2026-07-02 14:17:39.574428204 +0000 UTC m=+63.695670443" Jul 02 14:17:39.656374 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:39.656341 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:39.760010 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:39.759969 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:40.563049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.562976 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" event={"ID":"f98de5fa-8b99-409c-b217-eff19b9b536e","Type":"ContainerStarted","Data":"fbb7788f4b5ab5c607c9ddb01b649a0390664ef974c9c8f4d287fea5f6094639"} Jul 02 14:17:40.563686 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.563526 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:40.565015 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.564986 2581 patch_prober.go:28] interesting pod/console-operator-7fdbdf9c99-q6zxl container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.134.0.11:8443/readyz\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 02 14:17:40.565136 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.565036 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" containerName="console-operator" probeResult="failure" output="Get \"https://10.134.0.11:8443/readyz\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 02 14:17:40.566346 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.566315 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" event={"ID":"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33","Type":"ContainerStarted","Data":"493c0c1094b13415c2476546c34fa334e24ae3c0dc61f541aaffd260382e08f2"} Jul 02 14:17:40.581584 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.581417 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podStartSLOduration=0.747195605 podStartE2EDuration="3.581394251s" totalImagesPullingTime="2.834198646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:40.580483585 +0000 UTC m=+64.701725838" watchObservedRunningTime="2026-07-02 14:17:40.581394251 +0000 UTC m=+64.702636549" Jul 02 14:17:40.597427 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.597371 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" podStartSLOduration=0.781087069 podStartE2EDuration="3.59734951s" totalImagesPullingTime="2.816262441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:40.59657379 +0000 UTC m=+64.717816031" watchObservedRunningTime="2026-07-02 14:17:40.59734951 +0000 UTC m=+64.718591751" Jul 02 14:17:40.884479 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.884437 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:40.884667 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:40.884500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:40.884667 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:40.884600 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 14:17:40.884667 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:40.884612 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:17:40.884667 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:40.884631 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555b667bcf-7v5d7: secret "image-registry-tls" not found Jul 02 14:17:40.884667 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:40.884668 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls podName:0476e7a3-b391-499b-877d-e0402cacc30c nodeName:}" failed. No retries permitted until 2026-07-02 14:17:44.884648414 +0000 UTC m=+69.005890631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-hc7dp" (UID: "0476e7a3-b391-499b-877d-e0402cacc30c") : secret "samples-operator-tls" not found Jul 02 14:17:40.884923 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:40.884688 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls podName:8df61a8c-8b5d-495f-ab5c-0968f26ed55e nodeName:}" failed. No retries permitted until 2026-07-02 14:17:44.884678798 +0000 UTC m=+69.005921021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls") pod "image-registry-555b667bcf-7v5d7" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e") : secret "image-registry-tls" not found Jul 02 14:17:41.570799 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.570709 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" event={"ID":"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c","Type":"ContainerStarted","Data":"00812b9cdbc319496433de24ba35ff40bb40193855289fa4e4959a197fbc8321"} Jul 02 14:17:41.572385 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.572363 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/0.log" Jul 02 14:17:41.572500 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.572402 2581 generic.go:356] "Generic (PLEG): container finished" podID="f98de5fa-8b99-409c-b217-eff19b9b536e" containerID="fbb7788f4b5ab5c607c9ddb01b649a0390664ef974c9c8f4d287fea5f6094639" exitCode=255 Jul 02 14:17:41.572500 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.572460 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" event={"ID":"f98de5fa-8b99-409c-b217-eff19b9b536e","Type":"ContainerDied","Data":"fbb7788f4b5ab5c607c9ddb01b649a0390664ef974c9c8f4d287fea5f6094639"} Jul 02 14:17:41.572690 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.572675 2581 scope.go:117] "RemoveContainer" containerID="fbb7788f4b5ab5c607c9ddb01b649a0390664ef974c9c8f4d287fea5f6094639" Jul 02 14:17:41.589077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.589032 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" podStartSLOduration=1.977425489 podStartE2EDuration="4.589012557s" totalImagesPullingTime="2.611587068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:41.587922634 +0000 UTC m=+65.709164867" watchObservedRunningTime="2026-07-02 14:17:41.589012557 +0000 UTC m=+65.710254798" Jul 02 14:17:41.960869 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.960829 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-h4l85"] Jul 02 14:17:41.981788 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.981741 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h4l85"] Jul 02 14:17:41.981993 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.981837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:41.981993 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.981967 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:41.985021 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.985001 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 14:17:41.985148 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.985033 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 14:17:41.985266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.985239 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nfp8j\"" Jul 02 14:17:41.995373 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.995349 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:17:41.995488 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.995402 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:41.995488 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.995436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:17:41.995578 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:41.995499 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 14:17:41.995578 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:41.995560 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls podName:978d29b1-a9d0-44ee-95d2-11572d394c92 nodeName:}" failed. No retries permitted until 2026-07-02 14:18:13.995543807 +0000 UTC m=+98.116786026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls") pod "dns-default-tgp6n" (UID: "978d29b1-a9d0-44ee-95d2-11572d394c92") : secret "dns-default-metrics-tls" not found Jul 02 14:17:41.998047 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.998028 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 14:17:41.998141 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:41.998122 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 14:17:42.005818 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.005801 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 14:17:42.005923 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.005864 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs podName:7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3 nodeName:}" failed. No retries permitted until 2026-07-02 14:18:46.005843222 +0000 UTC m=+130.127085440 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs") pod "network-metrics-daemon-4mhz9" (UID: "7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3") : secret "metrics-daemon-secret" not found Jul 02 14:17:42.007875 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.007853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cdc7ec1e-5d10-4369-b9a3-0e0f77b38757-original-pull-secret\") pod \"global-pull-secret-syncer-dbr7t\" (UID: \"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757\") " pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:42.050488 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.050457 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:42.092915 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.092876 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dbr7t" Jul 02 14:17:42.096211 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.096191 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btlzm\" (UniqueName: \"kubernetes.io/projected/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-api-access-btlzm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.096269 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.096239 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.096321 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.096310 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eee82ccd-bc7b-420e-83df-9b148b9eff9d-crio-socket\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.096357 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.096339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:42.096390 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.096377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eee82ccd-bc7b-420e-83df-9b148b9eff9d-data-volume\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.096423 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.096410 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.098922 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.098905 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 14:17:42.109807 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.109786 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 14:17:42.120674 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.120652 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhzsq\" (UniqueName: \"kubernetes.io/projected/43db889f-6d33-4460-a927-c974412e98dd-kube-api-access-xhzsq\") pod \"network-check-target-8vhdd\" (UID: \"43db889f-6d33-4460-a927-c974412e98dd\") " pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:42.196889 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.196857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.197032 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.196953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eee82ccd-bc7b-420e-83df-9b148b9eff9d-crio-socket\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.197032 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.196980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eee82ccd-bc7b-420e-83df-9b148b9eff9d-data-volume\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.197032 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.197007 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.197176 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.197036 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eee82ccd-bc7b-420e-83df-9b148b9eff9d-crio-socket\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.197176 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.197053 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btlzm\" (UniqueName: \"kubernetes.io/projected/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-api-access-btlzm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.197176 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.196981 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:17:42.197315 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.197184 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:42.69714594 +0000 UTC m=+66.818388158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : configmap "kube-rbac-proxy" not found Jul 02 14:17:42.197315 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.197202 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:17:42.197315 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.197256 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:42.697240334 +0000 UTC m=+66.818482559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : secret "insights-runtime-extractor-tls" not found Jul 02 14:17:42.197421 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.197347 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eee82ccd-bc7b-420e-83df-9b148b9eff9d-data-volume\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.212454 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.212388 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btlzm\" (UniqueName: \"kubernetes.io/projected/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-api-access-btlzm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.261531 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.261498 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dbr7t"] Jul 02 14:17:42.264063 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.264040 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:42.269702 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:42.269677 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc7ec1e_5d10_4369_b9a3_0e0f77b38757.slice/crio-6ab79bc524647349ca212c290a418079357496f39dd3a4f4fcf4c1374c27d753 WatchSource:0}: Error finding container 6ab79bc524647349ca212c290a418079357496f39dd3a4f4fcf4c1374c27d753: Status 404 returned error can't find the container with id 6ab79bc524647349ca212c290a418079357496f39dd3a4f4fcf4c1374c27d753 Jul 02 14:17:42.353370 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.353331 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:42.402814 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.402787 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q6bk5\"" Jul 02 14:17:42.410900 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.410882 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:42.457506 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.457483 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:42.561719 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.561552 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8vhdd"] Jul 02 14:17:42.563911 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:17:42.563888 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43db889f_6d33_4460_a927_c974412e98dd.slice/crio-fadd1d06dc35ff95023b1c4a48be36e5d0d511c9f9d2aee87ef2377f699f9817 WatchSource:0}: Error finding container fadd1d06dc35ff95023b1c4a48be36e5d0d511c9f9d2aee87ef2377f699f9817: Status 404 returned error can't find the container with id fadd1d06dc35ff95023b1c4a48be36e5d0d511c9f9d2aee87ef2377f699f9817 Jul 02 14:17:42.576348 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.576321 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8vhdd" event={"ID":"43db889f-6d33-4460-a927-c974412e98dd","Type":"ContainerStarted","Data":"fadd1d06dc35ff95023b1c4a48be36e5d0d511c9f9d2aee87ef2377f699f9817"} Jul 02 14:17:42.577412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.577386 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dbr7t" event={"ID":"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757","Type":"ContainerStarted","Data":"6ab79bc524647349ca212c290a418079357496f39dd3a4f4fcf4c1374c27d753"} Jul 02 14:17:42.578891 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.578877 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/1.log" Jul 02 14:17:42.579328 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.579313 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/0.log" Jul 02 14:17:42.579394 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.579345 2581 generic.go:356] "Generic (PLEG): container finished" podID="f98de5fa-8b99-409c-b217-eff19b9b536e" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" exitCode=255 Jul 02 14:17:42.579429 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.579409 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" event={"ID":"f98de5fa-8b99-409c-b217-eff19b9b536e","Type":"ContainerDied","Data":"df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228"} Jul 02 14:17:42.579462 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.579445 2581 scope.go:117] "RemoveContainer" containerID="fbb7788f4b5ab5c607c9ddb01b649a0390664ef974c9c8f4d287fea5f6094639" Jul 02 14:17:42.579789 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.579769 2581 scope.go:117] "RemoveContainer" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" Jul 02 14:17:42.579933 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.579916 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:17:42.701314 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.701281 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.701515 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:42.701428 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:42.701712 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.701675 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:17:42.701712 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.701700 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:17:42.701866 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.701746 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:43.701726689 +0000 UTC m=+67.822968912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : secret "insights-runtime-extractor-tls" not found Jul 02 14:17:42.701866 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:42.701768 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:43.701759159 +0000 UTC m=+67.823001381 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : configmap "kube-rbac-proxy" not found Jul 02 14:17:43.585099 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:43.585073 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/1.log" Jul 02 14:17:43.585579 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:43.585494 2581 scope.go:117] "RemoveContainer" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" Jul 02 14:17:43.585958 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:43.585681 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:17:43.710185 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:43.710123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:43.710373 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:43.710255 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:17:43.710373 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:43.710328 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:45.710307614 +0000 UTC m=+69.831549855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : secret "insights-runtime-extractor-tls" not found Jul 02 14:17:43.710538 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:43.710431 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:43.710538 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:43.710520 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:17:43.710649 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:43.710567 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:45.710551245 +0000 UTC m=+69.831793479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : configmap "kube-rbac-proxy" not found Jul 02 14:17:43.725663 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:43.725571 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fsmwf_545cf0b4-dd6f-489f-8f52-3c697db25831/dns-node-resolver/0.log" Jul 02 14:17:44.921568 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:44.921519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:44.922097 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:44.921589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:44.922097 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:44.921689 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 14:17:44.922097 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:44.921767 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:17:44.922097 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:44.921787 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555b667bcf-7v5d7: secret "image-registry-tls" not found Jul 02 14:17:44.922097 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:44.921772 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls podName:0476e7a3-b391-499b-877d-e0402cacc30c nodeName:}" failed. No retries permitted until 2026-07-02 14:17:52.921751615 +0000 UTC m=+77.042993832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-hc7dp" (UID: "0476e7a3-b391-499b-877d-e0402cacc30c") : secret "samples-operator-tls" not found Jul 02 14:17:44.922097 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:44.921847 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls podName:8df61a8c-8b5d-495f-ab5c-0968f26ed55e nodeName:}" failed. No retries permitted until 2026-07-02 14:17:52.92183088 +0000 UTC m=+77.043073112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls") pod "image-registry-555b667bcf-7v5d7" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e") : secret "image-registry-tls" not found Jul 02 14:17:44.924636 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:44.924610 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kd2r7_2bffbb99-79da-4149-8564-b3b77a67615f/node-ca/0.log" Jul 02 14:17:45.724915 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:45.724873 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c9p6q_9a1b3fcc-b59c-480d-82eb-87a29f8cb07a/serve-healthcheck-canary/0.log" Jul 02 14:17:45.728110 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:45.728077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:45.728271 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:45.728210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:45.728271 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:45.728230 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:17:45.728383 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:45.728302 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:49.72828323 +0000 UTC m=+73.849525447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : configmap "kube-rbac-proxy" not found Jul 02 14:17:45.728383 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:45.728331 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:17:45.728383 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:45.728376 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:49.728365855 +0000 UTC m=+73.849608073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : secret "insights-runtime-extractor-tls" not found Jul 02 14:17:47.245610 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.245557 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:47.331649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.331613 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:47.410846 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.410818 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:47.475612 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.475579 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:47.475953 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.475941 2581 scope.go:117] "RemoveContainer" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" Jul 02 14:17:47.476096 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:47.476079 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:17:47.479540 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.479497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:47.598594 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.598497 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8vhdd" event={"ID":"43db889f-6d33-4460-a927-c974412e98dd","Type":"ContainerStarted","Data":"4ab8ed06a19bd96651c673a4bbdd8340ed499e8a62e591cecb769ea4c4ab0877"} Jul 02 14:17:47.598594 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.598574 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:17:47.599944 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.599917 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dbr7t" event={"ID":"cdc7ec1e-5d10-4369-b9a3-0e0f77b38757","Type":"ContainerStarted","Data":"7f3f157d2106fdf336311f5caefca6efd024b49b236bf92cf10158ed83f9d45b"} Jul 02 14:17:47.624664 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:47.624621 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8vhdd" podStartSLOduration=67.38294265 podStartE2EDuration="1m11.624608898s" totalImagesPullingTime="4.241666248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:47.623470565 +0000 UTC m=+71.744712828" watchObservedRunningTime="2026-07-02 14:17:47.624608898 +0000 UTC m=+71.745851137" Jul 02 14:17:48.418923 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:48.418890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:48.850092 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:48.850018 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:48.908797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:48.906108 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:49.137872 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:49.137836 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:49.236584 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:49.236545 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:49.763284 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:49.763247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:49.763654 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:49.763371 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 14:17:49.763654 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:49.763394 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:49.763654 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:49.763435 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:57.763419047 +0000 UTC m=+81.884661270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : configmap "kube-rbac-proxy" not found Jul 02 14:17:49.763654 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:49.763480 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:17:49.763654 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:49.763523 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:17:57.763511777 +0000 UTC m=+81.884753995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : secret "insights-runtime-extractor-tls" not found Jul 02 14:17:50.563881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:50.563850 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:17:50.564266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:50.564252 2581 scope.go:117] "RemoveContainer" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" Jul 02 14:17:50.564425 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:50.564406 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:17:50.767324 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:50.767288 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:51.696800 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:51.696764 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:52.629208 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:52.629177 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:52.750059 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:52.750023 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:52.915969 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:52.915886 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:52.991302 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:52.991267 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:17:52.991302 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:52.991305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:17:52.991509 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:52.991407 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 14:17:52.991509 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:52.991409 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 14:17:52.991509 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:52.991471 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls podName:0476e7a3-b391-499b-877d-e0402cacc30c nodeName:}" failed. No retries permitted until 2026-07-02 14:18:08.991456353 +0000 UTC m=+93.112698572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-hc7dp" (UID: "0476e7a3-b391-499b-877d-e0402cacc30c") : secret "samples-operator-tls" not found Jul 02 14:17:52.991509 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:52.991418 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555b667bcf-7v5d7: secret "image-registry-tls" not found Jul 02 14:17:52.991651 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:52.991536 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls podName:8df61a8c-8b5d-495f-ab5c-0968f26ed55e nodeName:}" failed. No retries permitted until 2026-07-02 14:18:08.99152494 +0000 UTC m=+93.112767161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls") pod "image-registry-555b667bcf-7v5d7" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e") : secret "image-registry-tls" not found Jul 02 14:17:53.053378 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:53.053345 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:53.153006 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:53.152972 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:53.826959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:53.826928 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:55.200363 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:55.200331 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:55.261932 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:55.261895 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:55.364772 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:55.364735 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:55.529939 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:55.529850 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:55.598414 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:55.598383 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:55.929508 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:55.929474 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:56.326772 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:56.326690 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:56.525900 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:56.525865 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:56.626804 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:56.626762 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:56.970406 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:56.970321 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:57.131041 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.131009 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:57.335784 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.335671 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:57.392997 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.392966 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:57.744728 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.744612 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:57.830340 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.830300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:57.830522 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.830409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:57.830588 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:57.830546 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 14:17:57.830640 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:17:57.830619 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls podName:eee82ccd-bc7b-420e-83df-9b148b9eff9d nodeName:}" failed. No retries permitted until 2026-07-02 14:18:13.830598376 +0000 UTC m=+97.951840611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h4l85" (UID: "eee82ccd-bc7b-420e-83df-9b148b9eff9d") : secret "insights-runtime-extractor-tls" not found Jul 02 14:17:57.830808 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.830787 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eee82ccd-bc7b-420e-83df-9b148b9eff9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:17:57.927643 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:57.927611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:58.319321 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:58.319280 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:59.686431 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:59.686397 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:59.785658 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:59.785623 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:17:59.923525 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:17:59.923493 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:00.134242 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:00.134208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:01.130062 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:01.130032 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:01.522098 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:01.522018 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:01.726839 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:01.726803 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:01.929957 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:01.929925 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:02.328959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:02.328878 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:02.921825 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:02.921784 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:03.006960 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.006934 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:03.144108 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.144069 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:03.280756 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.280685 2581 scope.go:117] "RemoveContainer" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" Jul 02 14:18:03.327163 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.327127 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:03.650362 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.650338 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:18:03.650811 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.650794 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/1.log" Jul 02 14:18:03.650861 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.650828 2581 generic.go:356] "Generic (PLEG): container finished" podID="f98de5fa-8b99-409c-b217-eff19b9b536e" containerID="91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c" exitCode=255 Jul 02 14:18:03.650861 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.650855 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" event={"ID":"f98de5fa-8b99-409c-b217-eff19b9b536e","Type":"ContainerDied","Data":"91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c"} Jul 02 14:18:03.650947 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.650896 2581 scope.go:117] "RemoveContainer" containerID="df210ef223c8f2d5cf366c5e5d3c57078d7d590170581f8aa9faa10e0639d228" Jul 02 14:18:03.651228 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.651212 2581 scope.go:117] "RemoveContainer" containerID="91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c" Jul 02 14:18:03.651401 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:03.651381 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:18:03.669629 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:03.669588 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dbr7t" podStartSLOduration=83.130651227 podStartE2EDuration="1m27.669573933s" totalImagesPullingTime="4.538922706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:17:47.648980002 +0000 UTC m=+71.770222242" watchObservedRunningTime="2026-07-02 14:18:03.669573933 +0000 UTC m=+87.790816164" Jul 02 14:18:04.538501 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:04.538476 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:04.602480 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:04.602453 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:04.655067 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:04.655040 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:18:07.476316 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:07.476285 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:18:07.476662 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:07.476589 2581 scope.go:117] "RemoveContainer" containerID="91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c" Jul 02 14:18:07.476751 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:07.476731 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:18:09.023628 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.023589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:18:09.023628 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.023632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:09.025901 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.025879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0476e7a3-b391-499b-877d-e0402cacc30c-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-hc7dp\" (UID: \"0476e7a3-b391-499b-877d-e0402cacc30c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:18:09.025972 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.025956 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"image-registry-555b667bcf-7v5d7\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:09.257335 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.257304 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" Jul 02 14:18:09.270036 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.270012 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:09.390757 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.390723 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp"] Jul 02 14:18:09.408330 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.408302 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555b667bcf-7v5d7"] Jul 02 14:18:09.411077 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:09.411053 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df61a8c_8b5d_495f_ab5c_0968f26ed55e.slice/crio-62c11bf548a69c482115804f5833b50e33463d5ec28ddd437886b3244515ef68 WatchSource:0}: Error finding container 62c11bf548a69c482115804f5833b50e33463d5ec28ddd437886b3244515ef68: Status 404 returned error can't find the container with id 62c11bf548a69c482115804f5833b50e33463d5ec28ddd437886b3244515ef68 Jul 02 14:18:09.670073 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.670033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" event={"ID":"0476e7a3-b391-499b-877d-e0402cacc30c","Type":"ContainerStarted","Data":"cafb6d285d87eb8f79b0294f032ad7fff534bf79ba2d6a505f689bcb87b52cd2"} Jul 02 14:18:09.671568 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.671527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" event={"ID":"8df61a8c-8b5d-495f-ab5c-0968f26ed55e","Type":"ContainerStarted","Data":"10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be"} Jul 02 14:18:09.671712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.671576 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" event={"ID":"8df61a8c-8b5d-495f-ab5c-0968f26ed55e","Type":"ContainerStarted","Data":"62c11bf548a69c482115804f5833b50e33463d5ec28ddd437886b3244515ef68"} Jul 02 14:18:09.671712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.671620 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:09.692486 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:09.692422 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" podStartSLOduration=32.692408471 podStartE2EDuration="32.692408471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:09.691645684 +0000 UTC m=+93.812887938" watchObservedRunningTime="2026-07-02 14:18:09.692408471 +0000 UTC m=+93.813650710" Jul 02 14:18:10.563532 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:10.563499 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:18:10.563931 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:10.563918 2581 scope.go:117] "RemoveContainer" containerID="91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c" Jul 02 14:18:10.564078 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:10.564058 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:18:11.604837 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.604791 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:11.678272 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.678244 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6"] Jul 02 14:18:11.702237 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.702208 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b"] Jul 02 14:18:11.702404 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.702365 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.705384 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.705354 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 14:18:11.705512 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.705411 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 02 14:18:11.705512 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.705411 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jmctx\"" Jul 02 14:18:11.706237 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.706220 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 14:18:11.706306 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.706260 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 14:18:11.725857 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.725828 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:11.728209 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.728190 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555b667bcf-7v5d7"] Jul 02 14:18:11.728280 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.728234 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6"] Jul 02 14:18:11.728280 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.728256 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b"] Jul 02 14:18:11.728363 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.728329 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.731961 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.731931 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 02 14:18:11.787425 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.787396 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b54fdc5c8-vzch2"] Jul 02 14:18:11.811622 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.811597 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.827923 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.827897 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b54fdc5c8-vzch2"] Jul 02 14:18:11.848203 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.848178 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srl6n\" (UniqueName: \"kubernetes.io/projected/82072878-3359-44fb-bb48-b59ab193ddea-kube-api-access-srl6n\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.848339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.848223 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr7bz\" (UniqueName: \"kubernetes.io/projected/e9021d92-93e2-4db4-884c-89e931e71bcb-kube-api-access-fr7bz\") pod \"managed-serviceaccount-addon-agent-766dfdd56-6hbf6\" (UID: \"e9021d92-93e2-4db4-884c-89e931e71bcb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.848339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.848265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9021d92-93e2-4db4-884c-89e931e71bcb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-766dfdd56-6hbf6\" (UID: \"e9021d92-93e2-4db4-884c-89e931e71bcb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.848339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.848300 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/82072878-3359-44fb-bb48-b59ab193ddea-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.848339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.848327 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82072878-3359-44fb-bb48-b59ab193ddea-tmp\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.868014 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.867938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:11.949097 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949077 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-image-registry-private-configuration\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949196 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srl6n\" (UniqueName: \"kubernetes.io/projected/82072878-3359-44fb-bb48-b59ab193ddea-kube-api-access-srl6n\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.949196 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949171 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-registry-tls\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949300 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr7bz\" (UniqueName: \"kubernetes.io/projected/e9021d92-93e2-4db4-884c-89e931e71bcb-kube-api-access-fr7bz\") pod \"managed-serviceaccount-addon-agent-766dfdd56-6hbf6\" (UID: \"e9021d92-93e2-4db4-884c-89e931e71bcb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.949300 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949224 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-bound-sa-token\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949300 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949246 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-registry-certificates\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949428 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qphbt\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-kube-api-access-qphbt\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949428 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9021d92-93e2-4db4-884c-89e931e71bcb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-766dfdd56-6hbf6\" (UID: \"e9021d92-93e2-4db4-884c-89e931e71bcb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.949428 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949396 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-ca-trust-extracted\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949428 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949421 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/82072878-3359-44fb-bb48-b59ab193ddea-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.949611 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949451 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82072878-3359-44fb-bb48-b59ab193ddea-tmp\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.949611 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949475 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-trusted-ca\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949611 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949496 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-installation-pull-secrets\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:11.949832 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.949804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82072878-3359-44fb-bb48-b59ab193ddea-tmp\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.951758 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.951739 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9021d92-93e2-4db4-884c-89e931e71bcb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-766dfdd56-6hbf6\" (UID: \"e9021d92-93e2-4db4-884c-89e931e71bcb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.952049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.952028 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/82072878-3359-44fb-bb48-b59ab193ddea-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.962959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.962931 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr7bz\" (UniqueName: \"kubernetes.io/projected/e9021d92-93e2-4db4-884c-89e931e71bcb-kube-api-access-fr7bz\") pod \"managed-serviceaccount-addon-agent-766dfdd56-6hbf6\" (UID: \"e9021d92-93e2-4db4-884c-89e931e71bcb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:11.963060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.963013 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srl6n\" (UniqueName: \"kubernetes.io/projected/82072878-3359-44fb-bb48-b59ab193ddea-kube-api-access-srl6n\") pod \"klusterlet-addon-workmgr-7f496c4d4f-nd55b\" (UID: \"82072878-3359-44fb-bb48-b59ab193ddea\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:11.967311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:11.967286 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:12.021231 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.021209 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" Jul 02 14:18:12.038086 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.038064 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:12.051106 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-bound-sa-token\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-registry-certificates\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051295 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qphbt\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-kube-api-access-qphbt\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-ca-trust-extracted\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-trusted-ca\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051453 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-installation-pull-secrets\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-image-registry-private-configuration\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.051681 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-registry-tls\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.052087 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.051865 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-ca-trust-extracted\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.052439 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.052386 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-registry-certificates\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.053114 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.053073 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-trusted-ca\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.054961 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.054881 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-installation-pull-secrets\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.055127 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.055108 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-registry-tls\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.055535 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.055515 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-image-registry-private-configuration\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.062869 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.062728 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:12.063873 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.063853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qphbt\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-kube-api-access-qphbt\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.064448 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.064429 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0bbc3c98-b1d0-41df-a195-4fe50fb1ad67-bound-sa-token\") pod \"image-registry-b54fdc5c8-vzch2\" (UID: \"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67\") " pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.120596 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.120305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.186477 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.186318 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6"] Jul 02 14:18:12.190099 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:12.190057 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9021d92_93e2_4db4_884c_89e931e71bcb.slice/crio-bbd9a29c5df0519bb2992f52b7bc1ab2d0f3b2fe4d262c7af585c4a54b486264 WatchSource:0}: Error finding container bbd9a29c5df0519bb2992f52b7bc1ab2d0f3b2fe4d262c7af585c4a54b486264: Status 404 returned error can't find the container with id bbd9a29c5df0519bb2992f52b7bc1ab2d0f3b2fe4d262c7af585c4a54b486264 Jul 02 14:18:12.213486 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.213408 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b"] Jul 02 14:18:12.218754 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.218614 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:12.293401 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.293351 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b54fdc5c8-vzch2"] Jul 02 14:18:12.295791 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:12.295760 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbc3c98_b1d0_41df_a195_4fe50fb1ad67.slice/crio-22bc9a53dcffd88f9a37a09a4916982adeb0a8e230123f4602e72320b63e75bc WatchSource:0}: Error finding container 22bc9a53dcffd88f9a37a09a4916982adeb0a8e230123f4602e72320b63e75bc: Status 404 returned error can't find the container with id 22bc9a53dcffd88f9a37a09a4916982adeb0a8e230123f4602e72320b63e75bc Jul 02 14:18:12.682674 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.682592 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" event={"ID":"e9021d92-93e2-4db4-884c-89e931e71bcb","Type":"ContainerStarted","Data":"bbd9a29c5df0519bb2992f52b7bc1ab2d0f3b2fe4d262c7af585c4a54b486264"} Jul 02 14:18:12.684463 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.684434 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" event={"ID":"0476e7a3-b391-499b-877d-e0402cacc30c","Type":"ContainerStarted","Data":"d364fa9811922dccd44822040270f75badee034c3deed0d08bb38f53ff2957c5"} Jul 02 14:18:12.684577 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.684522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" event={"ID":"0476e7a3-b391-499b-877d-e0402cacc30c","Type":"ContainerStarted","Data":"6810af26c45e13b00b47c92ad335ac869a860cfa5f720f3c57a4fb733e231f19"} Jul 02 14:18:12.685917 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.685891 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" event={"ID":"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67","Type":"ContainerStarted","Data":"38eea9ab8c2a035d83426966e734931687ce7946948723f1bf5acf052a4412b0"} Jul 02 14:18:12.686013 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.685934 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" event={"ID":"0bbc3c98-b1d0-41df-a195-4fe50fb1ad67","Type":"ContainerStarted","Data":"22bc9a53dcffd88f9a37a09a4916982adeb0a8e230123f4602e72320b63e75bc"} Jul 02 14:18:12.686079 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.686049 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:12.687030 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.687004 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" event={"ID":"82072878-3359-44fb-bb48-b59ab193ddea","Type":"ContainerStarted","Data":"4e192d80f8c041e580e8468dd4c92587baed517c6aa47ab47c214ccc6625f228"} Jul 02 14:18:12.702743 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.702700 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-hc7dp" podStartSLOduration=33.220105622 podStartE2EDuration="35.70268755s" totalImagesPullingTime="2.482581928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:12.700922268 +0000 UTC m=+96.822164507" watchObservedRunningTime="2026-07-02 14:18:12.70268755 +0000 UTC m=+96.823929790" Jul 02 14:18:12.720705 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:12.720661 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" podStartSLOduration=1.720646526 podStartE2EDuration="1.720646526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:12.719933966 +0000 UTC m=+96.841176207" watchObservedRunningTime="2026-07-02 14:18:12.720646526 +0000 UTC m=+96.841888744" Jul 02 14:18:13.867676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:13.867636 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:18:13.870827 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:13.870795 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eee82ccd-bc7b-420e-83df-9b148b9eff9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h4l85\" (UID: \"eee82ccd-bc7b-420e-83df-9b148b9eff9d\") " pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:18:14.069550 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.069517 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:18:14.072548 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.072492 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/978d29b1-a9d0-44ee-95d2-11572d394c92-metrics-tls\") pod \"dns-default-tgp6n\" (UID: \"978d29b1-a9d0-44ee-95d2-11572d394c92\") " pod="openshift-dns/dns-default-tgp6n" Jul 02 14:18:14.080908 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.080716 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mttc5\"" Jul 02 14:18:14.089430 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.089225 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tgp6n" Jul 02 14:18:14.090558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.090535 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h4l85" Jul 02 14:18:14.287704 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.287381 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tgp6n"] Jul 02 14:18:14.304266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:14.304228 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h4l85"] Jul 02 14:18:15.021813 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:15.021781 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978d29b1_a9d0_44ee_95d2_11572d394c92.slice/crio-f81ca8c80dacfd27d398ae7f2313da76487d90abcc6bc19e92fcfe0e6fdc088c WatchSource:0}: Error finding container f81ca8c80dacfd27d398ae7f2313da76487d90abcc6bc19e92fcfe0e6fdc088c: Status 404 returned error can't find the container with id f81ca8c80dacfd27d398ae7f2313da76487d90abcc6bc19e92fcfe0e6fdc088c Jul 02 14:18:15.348372 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:15.348309 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee82ccd_bc7b_420e_83df_9b148b9eff9d.slice/crio-a11dcad4963610b71bc17b9ef78530da4b45a231f1beb787c43f895b3e9e5176 WatchSource:0}: Error finding container a11dcad4963610b71bc17b9ef78530da4b45a231f1beb787c43f895b3e9e5176: Status 404 returned error can't find the container with id a11dcad4963610b71bc17b9ef78530da4b45a231f1beb787c43f895b3e9e5176 Jul 02 14:18:15.421241 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:15.421208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:15.522272 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:15.522245 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:15.700821 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:15.700785 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tgp6n" event={"ID":"978d29b1-a9d0-44ee-95d2-11572d394c92","Type":"ContainerStarted","Data":"f81ca8c80dacfd27d398ae7f2313da76487d90abcc6bc19e92fcfe0e6fdc088c"} Jul 02 14:18:15.702148 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:15.702108 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4l85" event={"ID":"eee82ccd-bc7b-420e-83df-9b148b9eff9d","Type":"ContainerStarted","Data":"a11dcad4963610b71bc17b9ef78530da4b45a231f1beb787c43f895b3e9e5176"} Jul 02 14:18:16.710432 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:16.710382 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" event={"ID":"82072878-3359-44fb-bb48-b59ab193ddea","Type":"ContainerStarted","Data":"3787a87f9769061b242d893c3d7f07878a0d66294e96c7dd36c57404f3299bce"} Jul 02 14:18:16.710902 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:16.710564 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:16.712700 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:16.712624 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" Jul 02 14:18:16.714065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:16.714040 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" event={"ID":"e9021d92-93e2-4db4-884c-89e931e71bcb","Type":"ContainerStarted","Data":"179cd5a5ca23401f7c114fd4180c3af1fa2f248af8b1a64c463ced6efab3dbf9"} Jul 02 14:18:16.728923 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:16.728870 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f496c4d4f-nd55b" podStartSLOduration=1.420423508 podStartE2EDuration="5.728849689s" totalImagesPullingTime="4.308426181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:18:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:16.726980984 +0000 UTC m=+100.848223223" watchObservedRunningTime="2026-07-02 14:18:16.728849689 +0000 UTC m=+100.850091926" Jul 02 14:18:16.747205 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:16.747132 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-766dfdd56-6hbf6" podStartSLOduration=1.423380065 podStartE2EDuration="5.747117193s" totalImagesPullingTime="4.323737128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:18:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:16.74680661 +0000 UTC m=+100.868048852" watchObservedRunningTime="2026-07-02 14:18:16.747117193 +0000 UTC m=+100.868359434" Jul 02 14:18:18.605782 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:18.605747 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8vhdd" Jul 02 14:18:18.725754 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:18.725713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tgp6n" event={"ID":"978d29b1-a9d0-44ee-95d2-11572d394c92","Type":"ContainerStarted","Data":"bffa10a10c752fa946ff4bc6c003729009393fe6c426255b77c2782bd08f93fa"} Jul 02 14:18:18.725914 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:18.725783 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tgp6n" event={"ID":"978d29b1-a9d0-44ee-95d2-11572d394c92","Type":"ContainerStarted","Data":"028c633fcd15348c42fd0da1484196f18140ab9d2e9068ea777ac436e490edeb"} Jul 02 14:18:18.746032 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:18.745983 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tgp6n" podStartSLOduration=65.837702745 podStartE2EDuration="1m8.745966255s" totalImagesPullingTime="2.90826351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:17:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:18.744467623 +0000 UTC m=+102.865709864" watchObservedRunningTime="2026-07-02 14:18:18.745966255 +0000 UTC m=+102.867208495" Jul 02 14:18:19.536879 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:19.536843 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:19.598167 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:19.598117 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:19.729216 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:19.729190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tgp6n" Jul 02 14:18:19.982080 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:19.982042 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:21.737531 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:21.737492 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4l85" event={"ID":"eee82ccd-bc7b-420e-83df-9b148b9eff9d","Type":"ContainerStarted","Data":"628326602e19b82e85a994b99e4c01399ef195e83ce04f291ca9974f8df022a2"} Jul 02 14:18:22.158504 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.158476 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:22.286108 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.286081 2581 scope.go:117] "RemoveContainer" containerID="91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c" Jul 02 14:18:22.286278 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:22.286240 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6zxl_openshift-console-operator(f98de5fa-8b99-409c-b217-eff19b9b536e)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" podUID="f98de5fa-8b99-409c-b217-eff19b9b536e" Jul 02 14:18:22.397225 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.397196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:22.448481 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.448402 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t8tzl"] Jul 02 14:18:22.450709 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.450691 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.454501 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.454480 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 14:18:22.454632 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.454524 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 14:18:22.455881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.455747 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 14:18:22.455881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.455759 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 14:18:22.455881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.455765 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h5vwt\"" Jul 02 14:18:22.455881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.455777 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 14:18:22.455881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.455765 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 14:18:22.486058 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.486022 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:22.643303 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643273 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-tls\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643303 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-sys\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-textfile\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643342 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643367 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh7sd\" (UniqueName: \"kubernetes.io/projected/08c9600e-0707-4326-8ffa-9d12fd349f54-kube-api-access-rh7sd\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-root\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08c9600e-0707-4326-8ffa-9d12fd349f54-metrics-client-ca\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643507 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-wtmp\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.643680 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.643510 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-accelerators-collector-config\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.743437 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.743358 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4l85" event={"ID":"eee82ccd-bc7b-420e-83df-9b148b9eff9d","Type":"ContainerStarted","Data":"f103ac76e7b4aefbb1c04281713a8ed6a6bb824fcfa9e0cd72825ea55080c7af"} Jul 02 14:18:22.744692 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-wtmp\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744787 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-accelerators-collector-config\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744787 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744774 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-tls\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-sys\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-textfile\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744848 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744852 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-wtmp\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744889 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-sys\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:22.744904 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh7sd\" (UniqueName: \"kubernetes.io/projected/08c9600e-0707-4326-8ffa-9d12fd349f54-kube-api-access-rh7sd\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.744959 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-root\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.745382 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:22.744978 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-tls podName:08c9600e-0707-4326-8ffa-9d12fd349f54 nodeName:}" failed. No retries permitted until 2026-07-02 14:18:23.244957622 +0000 UTC m=+107.366199842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-tls") pod "node-exporter-t8tzl" (UID: "08c9600e-0707-4326-8ffa-9d12fd349f54") : secret "node-exporter-tls" not found Jul 02 14:18:22.745382 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.745014 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08c9600e-0707-4326-8ffa-9d12fd349f54-metrics-client-ca\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.745382 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.744978 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08c9600e-0707-4326-8ffa-9d12fd349f54-root\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.745382 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.745246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-textfile\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.745588 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.745419 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-accelerators-collector-config\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.745588 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.745496 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08c9600e-0707-4326-8ffa-9d12fd349f54-metrics-client-ca\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.747290 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.747267 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:22.755082 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:22.755059 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh7sd\" (UniqueName: \"kubernetes.io/projected/08c9600e-0707-4326-8ffa-9d12fd349f54-kube-api-access-rh7sd\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:23.116052 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.115964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:23.249231 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.249196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-tls\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:23.251710 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.251684 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08c9600e-0707-4326-8ffa-9d12fd349f54-node-exporter-tls\") pod \"node-exporter-t8tzl\" (UID: \"08c9600e-0707-4326-8ffa-9d12fd349f54\") " pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:23.315102 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.315052 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:23.359149 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.359121 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t8tzl" Jul 02 14:18:23.369837 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:23.369806 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c9600e_0707_4326_8ffa_9d12fd349f54.slice/crio-0f8d67ab85f3cd3f400a1aeae9b68ddae374ec7162bc91d441b21aef12107ce3 WatchSource:0}: Error finding container 0f8d67ab85f3cd3f400a1aeae9b68ddae374ec7162bc91d441b21aef12107ce3: Status 404 returned error can't find the container with id 0f8d67ab85f3cd3f400a1aeae9b68ddae374ec7162bc91d441b21aef12107ce3 Jul 02 14:18:23.459501 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.459450 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:23.572394 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.572352 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:23.666499 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.666446 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:23.747933 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.747896 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8tzl" event={"ID":"08c9600e-0707-4326-8ffa-9d12fd349f54","Type":"ContainerStarted","Data":"0f8d67ab85f3cd3f400a1aeae9b68ddae374ec7162bc91d441b21aef12107ce3"} Jul 02 14:18:23.998528 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:23.998428 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:25.397386 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:25.397344 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:25.495963 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:25.495932 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:25.574758 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:25.574722 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:25.759980 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:25.759956 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h4l85" event={"ID":"eee82ccd-bc7b-420e-83df-9b148b9eff9d","Type":"ContainerStarted","Data":"a90f11a43229a3641553b9efe3e486452c0a4deb06344abd1b6a170414381fed"} Jul 02 14:18:25.770737 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:25.770707 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:25.781016 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:25.780977 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-h4l85" podStartSLOduration=35.573087698 podStartE2EDuration="44.780963694s" totalImagesPullingTime="9.207875996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 14:17:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:25.780330862 +0000 UTC m=+109.901573102" watchObservedRunningTime="2026-07-02 14:18:25.780963694 +0000 UTC m=+109.902205934" Jul 02 14:18:26.764790 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:26.764754 2581 generic.go:356] "Generic (PLEG): container finished" podID="08c9600e-0707-4326-8ffa-9d12fd349f54" containerID="f479657a728c44a620fafc57a1c004add05f8cee4befe203fde5c123220fb648" exitCode=0 Jul 02 14:18:26.765270 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:26.764838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8tzl" event={"ID":"08c9600e-0707-4326-8ffa-9d12fd349f54","Type":"ContainerDied","Data":"f479657a728c44a620fafc57a1c004add05f8cee4befe203fde5c123220fb648"} Jul 02 14:18:27.187995 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:27.187964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:27.772614 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:27.772585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8tzl" event={"ID":"08c9600e-0707-4326-8ffa-9d12fd349f54","Type":"ContainerStarted","Data":"9bd29f899242d14b97c28197d78ebfbfa56115dad0320611629f0069c0e78a77"} Jul 02 14:18:27.772971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:27.772633 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t8tzl" event={"ID":"08c9600e-0707-4326-8ffa-9d12fd349f54","Type":"ContainerStarted","Data":"f93349f01df87de3a4f9771479ea6e3d1012ba9c215de65579dc8353cc720926"} Jul 02 14:18:27.799287 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:27.799239 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t8tzl" podStartSLOduration=3.3998003470000002 podStartE2EDuration="5.799223229s" totalImagesPullingTime="2.335006926s" totalInitContainerRuntime="64.415956ms" isStatefulPod=true podCreationTimestamp="2026-07-02 14:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:27.794773463 +0000 UTC m=+111.916015706" watchObservedRunningTime="2026-07-02 14:18:27.799223229 +0000 UTC m=+111.920465468" Jul 02 14:18:27.808100 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:27.808078 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:28.671133 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:28.671098 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:29.735753 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:29.735726 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tgp6n" Jul 02 14:18:31.734304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:31.734277 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:32.124813 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:32.124719 2581 patch_prober.go:28] interesting pod/image-registry-b54fdc5c8-vzch2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 14:18:32.124813 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:32.124783 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" podUID="0bbc3c98-b1d0-41df-a195-4fe50fb1ad67" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:18:33.462099 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:33.462065 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:33.563395 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:33.563369 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:33.696907 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:33.696882 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b54fdc5c8-vzch2" Jul 02 14:18:33.883956 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:33.883924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:33.984490 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:33.984463 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:34.281582 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:34.281506 2581 scope.go:117] "RemoveContainer" containerID="91ee2d88972e35571adc7e615d9b1d7a6dfea11c558513656786f2f5d543749c" Jul 02 14:18:34.798280 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:34.798254 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:18:34.798668 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:34.798356 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" event={"ID":"f98de5fa-8b99-409c-b217-eff19b9b536e","Type":"ContainerStarted","Data":"b8f19803e477b9c722b3b8836100268439d3c0b4c45d81aee78e6ea097127abc"} Jul 02 14:18:34.798668 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:34.798629 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:18:34.973717 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:34.973689 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6zxl" Jul 02 14:18:35.175884 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:35.175851 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:35.273256 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:35.273226 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:35.376796 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:35.376766 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:35.476193 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:35.476103 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:35.574101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:35.574067 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:35.674135 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:35.674104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.039435 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.039401 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.125555 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.125522 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.255004 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.254956 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.434978 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.434943 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.495287 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.495253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.748727 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.748640 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" podUID="8df61a8c-8b5d-495f-ab5c-0968f26ed55e" containerName="registry" containerID="cri-o://10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be" gracePeriod=30 Jul 02 14:18:36.852530 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.852498 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:36.983078 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:36.983058 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:37.057181 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057085 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-certificates\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057181 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057124 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057181 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057170 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-image-registry-private-configuration\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057706 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057211 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-trusted-ca\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057706 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057245 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q8vq\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-kube-api-access-9q8vq\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057706 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057274 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-ca-trust-extracted\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057706 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057308 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-bound-sa-token\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057706 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057345 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-installation-pull-secrets\") pod \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\" (UID: \"8df61a8c-8b5d-495f-ab5c-0968f26ed55e\") " Jul 02 14:18:37.057706 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.057658 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:37.058396 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.058366 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:18:37.059946 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.059917 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-kube-api-access-9q8vq" (OuterVolumeSpecName: "kube-api-access-9q8vq") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "kube-api-access-9q8vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:37.059946 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.059936 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:37.060083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.059961 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:37.060083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.059970 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:18:37.060083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.060004 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:18:37.067034 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.067011 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8df61a8c-8b5d-495f-ab5c-0968f26ed55e" (UID: "8df61a8c-8b5d-495f-ab5c-0968f26ed55e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:18:37.158868 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158835 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.158868 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158864 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-image-registry-private-configuration\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.158868 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158874 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-trusted-ca\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.159074 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158882 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q8vq\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-kube-api-access-9q8vq\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.159074 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158892 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-ca-trust-extracted\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.159074 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158900 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-bound-sa-token\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.159074 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158908 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-installation-pull-secrets\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.159074 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.158916 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8df61a8c-8b5d-495f-ab5c-0968f26ed55e-registry-certificates\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:18:37.177815 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.177787 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:37.761602 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.761570 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:37.810106 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.810081 2581 generic.go:356] "Generic (PLEG): container finished" podID="8df61a8c-8b5d-495f-ab5c-0968f26ed55e" containerID="10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be" exitCode=0 Jul 02 14:18:37.810251 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.810128 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" Jul 02 14:18:37.810251 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.810130 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" event={"ID":"8df61a8c-8b5d-495f-ab5c-0968f26ed55e","Type":"ContainerDied","Data":"10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be"} Jul 02 14:18:37.810323 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.810276 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555b667bcf-7v5d7" event={"ID":"8df61a8c-8b5d-495f-ab5c-0968f26ed55e","Type":"ContainerDied","Data":"62c11bf548a69c482115804f5833b50e33463d5ec28ddd437886b3244515ef68"} Jul 02 14:18:37.810323 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.810303 2581 scope.go:117] "RemoveContainer" containerID="10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be" Jul 02 14:18:37.819945 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.819852 2581 scope.go:117] "RemoveContainer" containerID="10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be" Jul 02 14:18:37.820391 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:18:37.820286 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be\": container with ID starting with 10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be not found: ID does not exist" containerID="10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be" Jul 02 14:18:37.820391 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.820336 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be"} err="failed to get container status \"10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be\": rpc error: code = NotFound desc = could not find container \"10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be\": container with ID starting with 10da80114df6251482d12b5356165de9d8a1e9234840665eb396ade95df170be not found: ID does not exist" Jul 02 14:18:37.835269 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.835244 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555b667bcf-7v5d7"] Jul 02 14:18:37.842918 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.842898 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-555b667bcf-7v5d7"] Jul 02 14:18:37.872993 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.872973 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:37.974083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:37.974053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:38.284397 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:38.284367 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8df61a8c-8b5d-495f-ab5c-0968f26ed55e" path="/var/lib/kubelet/pods/8df61a8c-8b5d-495f-ab5c-0968f26ed55e/volumes" Jul 02 14:18:40.158062 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:40.158031 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:40.259812 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:40.259774 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:41.680850 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.680813 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-645c47d5c6-8br79"] Jul 02 14:18:41.681248 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.681053 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df61a8c-8b5d-495f-ab5c-0968f26ed55e" containerName="registry" Jul 02 14:18:41.681248 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.681063 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df61a8c-8b5d-495f-ab5c-0968f26ed55e" containerName="registry" Jul 02 14:18:41.681248 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.681112 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df61a8c-8b5d-495f-ab5c-0968f26ed55e" containerName="registry" Jul 02 14:18:41.684988 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.684970 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.687527 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.687506 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 14:18:41.687646 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.687565 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 14:18:41.687646 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.687568 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 14:18:41.688764 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.688743 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 14:18:41.688912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.688799 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 14:18:41.688912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.688813 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 14:18:41.688912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.688802 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 14:18:41.688912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.688801 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-47g6m\"" Jul 02 14:18:41.693226 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.693206 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645c47d5c6-8br79"] Jul 02 14:18:41.798870 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.798833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-oauth-config\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.799030 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.798865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-service-ca\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.799030 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.798935 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-oauth-serving-cert\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.799030 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.798953 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfb46\" (UniqueName: \"kubernetes.io/projected/41945543-1d11-4e3b-9052-ce1836545a02-kube-api-access-pfb46\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.799030 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.798997 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-console-config\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.799191 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.799046 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-serving-cert\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.900262 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.900227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-serving-cert\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.900434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.900284 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-oauth-config\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.900434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.900317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-service-ca\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.900434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.900351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-oauth-serving-cert\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.900434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.900373 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfb46\" (UniqueName: \"kubernetes.io/projected/41945543-1d11-4e3b-9052-ce1836545a02-kube-api-access-pfb46\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.900434 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.900425 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-console-config\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.901194 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.901149 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-oauth-serving-cert\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.901287 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.901238 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-service-ca\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.901287 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.901244 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-console-config\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.902653 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.902634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-oauth-config\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.902741 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.902713 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-serving-cert\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.909273 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.909252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfb46\" (UniqueName: \"kubernetes.io/projected/41945543-1d11-4e3b-9052-ce1836545a02-kube-api-access-pfb46\") pod \"console-645c47d5c6-8br79\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:41.994275 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:41.994192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:42.124675 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:42.124604 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-645c47d5c6-8br79"] Jul 02 14:18:42.127193 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:42.127147 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41945543_1d11_4e3b_9052_ce1836545a02.slice/crio-1f392730d449aad25c9e679f239674900557968b3f98bf7458a5fed3974bf975 WatchSource:0}: Error finding container 1f392730d449aad25c9e679f239674900557968b3f98bf7458a5fed3974bf975: Status 404 returned error can't find the container with id 1f392730d449aad25c9e679f239674900557968b3f98bf7458a5fed3974bf975 Jul 02 14:18:42.470447 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:42.470416 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:42.570765 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:42.570732 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:42.829473 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:42.829401 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c47d5c6-8br79" event={"ID":"41945543-1d11-4e3b-9052-ce1836545a02","Type":"ContainerStarted","Data":"1f392730d449aad25c9e679f239674900557968b3f98bf7458a5fed3974bf975"} Jul 02 14:18:44.477502 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:44.477464 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:45.047989 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:45.047960 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:46.038047 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.038000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:18:46.040912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.040880 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3-metrics-certs\") pod \"network-metrics-daemon-4mhz9\" (UID: \"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3\") " pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:18:46.120920 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.120893 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:46.224487 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.224454 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:46.308171 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.308080 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8mqz5\"" Jul 02 14:18:46.316215 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.316198 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mhz9" Jul 02 14:18:46.443500 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.443474 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4mhz9"] Jul 02 14:18:46.447815 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:46.447785 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e05eeb2_39fa_4a9b_8d4e_22bc10dadfa3.slice/crio-bcd94f10c0c75beee14583ccafb40692ba805a405f564e48c0ee52b9b8237f8b WatchSource:0}: Error finding container bcd94f10c0c75beee14583ccafb40692ba805a405f564e48c0ee52b9b8237f8b: Status 404 returned error can't find the container with id bcd94f10c0c75beee14583ccafb40692ba805a405f564e48c0ee52b9b8237f8b Jul 02 14:18:46.476563 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.476535 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:46.553325 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.553295 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:46.845275 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.845240 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mhz9" event={"ID":"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3","Type":"ContainerStarted","Data":"bcd94f10c0c75beee14583ccafb40692ba805a405f564e48c0ee52b9b8237f8b"} Jul 02 14:18:46.846689 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.846657 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c47d5c6-8br79" event={"ID":"41945543-1d11-4e3b-9052-ce1836545a02","Type":"ContainerStarted","Data":"a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3"} Jul 02 14:18:46.866040 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:46.865966 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-645c47d5c6-8br79" podStartSLOduration=1.884908706 podStartE2EDuration="5.865951354s" totalImagesPullingTime="3.981042648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:46.864951974 +0000 UTC m=+130.986194215" watchObservedRunningTime="2026-07-02 14:18:46.865951354 +0000 UTC m=+130.987193594" Jul 02 14:18:49.542351 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:49.542320 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:49.645212 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:49.645183 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:50.273614 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:50.273580 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:50.374858 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:50.374804 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:50.861616 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:50.861579 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mhz9" event={"ID":"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3","Type":"ContainerStarted","Data":"2b6013679dabd5a203971fe933afa92021d8a64bc1a80edc3a1ac6d3b2b7c9b0"} Jul 02 14:18:50.862001 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:50.861640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mhz9" event={"ID":"7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3","Type":"ContainerStarted","Data":"2221f744c72774a9ec8187748e4da0c2427445e9ae28dd881c3d7636ee57ff35"} Jul 02 14:18:50.879696 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:50.879648 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4mhz9" podStartSLOduration=131.211570418 podStartE2EDuration="2m14.879633505s" totalImagesPullingTime="3.668063087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:50.878110863 +0000 UTC m=+134.999353116" watchObservedRunningTime="2026-07-02 14:18:50.879633505 +0000 UTC m=+135.000875744" Jul 02 14:18:50.980232 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:50.980193 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:51.042901 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.042866 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:51.068146 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.068116 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75f66dbf88-7tjvf"] Jul 02 14:18:51.085627 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.085603 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75f66dbf88-7tjvf"] Jul 02 14:18:51.085780 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.085750 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.092887 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.092864 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 14:18:51.178743 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.178716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-serving-cert\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.178938 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.178758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-oauth-config\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.178938 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.178778 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-console-config\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.178938 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.178871 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-trusted-ca-bundle\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.179085 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.178933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrfv4\" (UniqueName: \"kubernetes.io/projected/5a684867-c209-4781-94b1-af0f470b9ee6-kube-api-access-zrfv4\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.179085 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.178978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-service-ca\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.179085 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.179016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-oauth-serving-cert\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.279904 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.279874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrfv4\" (UniqueName: \"kubernetes.io/projected/5a684867-c209-4781-94b1-af0f470b9ee6-kube-api-access-zrfv4\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.279911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-service-ca\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.279931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-oauth-serving-cert\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.279963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-serving-cert\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.279997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-oauth-config\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.280013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-console-config\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.280047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-trusted-ca-bundle\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280728 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.280706 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-oauth-serving-cert\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280823 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.280748 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-service-ca\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.280818 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-console-config\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.280941 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.280928 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-trusted-ca-bundle\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.282428 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.282409 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-serving-cert\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.282535 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.282516 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-oauth-config\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.289679 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.289659 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrfv4\" (UniqueName: \"kubernetes.io/projected/5a684867-c209-4781-94b1-af0f470b9ee6-kube-api-access-zrfv4\") pod \"console-75f66dbf88-7tjvf\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.395874 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.395831 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:18:51.568368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.568333 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75f66dbf88-7tjvf"] Jul 02 14:18:51.570981 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:18:51.570952 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a684867_c209_4781_94b1_af0f470b9ee6.slice/crio-6bc6da890304b21553ecd9789f535f90cb859be92be89aa96868b7bc978030ec WatchSource:0}: Error finding container 6bc6da890304b21553ecd9789f535f90cb859be92be89aa96868b7bc978030ec: Status 404 returned error can't find the container with id 6bc6da890304b21553ecd9789f535f90cb859be92be89aa96868b7bc978030ec Jul 02 14:18:51.866804 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.866717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f66dbf88-7tjvf" event={"ID":"5a684867-c209-4781-94b1-af0f470b9ee6","Type":"ContainerStarted","Data":"a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c"} Jul 02 14:18:51.866804 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.866775 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f66dbf88-7tjvf" event={"ID":"5a684867-c209-4781-94b1-af0f470b9ee6","Type":"ContainerStarted","Data":"6bc6da890304b21553ecd9789f535f90cb859be92be89aa96868b7bc978030ec"} Jul 02 14:18:51.885500 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.885444 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75f66dbf88-7tjvf" podStartSLOduration=0.885429136 podStartE2EDuration="885.429136ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:18:51.885089799 +0000 UTC m=+136.006332053" watchObservedRunningTime="2026-07-02 14:18:51.885429136 +0000 UTC m=+136.006671375" Jul 02 14:18:51.994447 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.994413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:51.994632 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.994471 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:51.999308 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:51.999286 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:52.874503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:52.874475 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:18:53.875911 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:53.875872 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:53.977503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:53.977468 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:56.891813 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:56.891778 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e2045a4-0e5b-483b-9f85-43d1ab8b8d33" containerID="493c0c1094b13415c2476546c34fa334e24ae3c0dc61f541aaffd260382e08f2" exitCode=0 Jul 02 14:18:56.892205 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:56.891821 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" event={"ID":"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33","Type":"ContainerDied","Data":"493c0c1094b13415c2476546c34fa334e24ae3c0dc61f541aaffd260382e08f2"} Jul 02 14:18:56.892279 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:56.892213 2581 scope.go:117] "RemoveContainer" containerID="493c0c1094b13415c2476546c34fa334e24ae3c0dc61f541aaffd260382e08f2" Jul 02 14:18:56.958140 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:56.958116 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:57.240897 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:57.240814 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:57.341463 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:57.341426 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:18:57.896642 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:18:57.896610 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-fsmx9" event={"ID":"3e2045a4-0e5b-483b-9f85-43d1ab8b8d33","Type":"ContainerStarted","Data":"11945f9609f6997cdc22c627a4a0e1b270773cc602a837dc55f0a512e89f1311"} Jul 02 14:19:00.682092 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:00.682059 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:00.783335 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:00.783302 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:01.396505 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.396465 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:19:01.396688 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.396542 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:19:01.401374 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.401350 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:19:01.914630 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.914593 2581 generic.go:356] "Generic (PLEG): container finished" podID="87a7e4f0-15dd-4ee3-bf66-9a0033fa496c" containerID="00812b9cdbc319496433de24ba35ff40bb40193855289fa4e4959a197fbc8321" exitCode=0 Jul 02 14:19:01.915031 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.914664 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" event={"ID":"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c","Type":"ContainerDied","Data":"00812b9cdbc319496433de24ba35ff40bb40193855289fa4e4959a197fbc8321"} Jul 02 14:19:01.915145 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.915129 2581 scope.go:117] "RemoveContainer" containerID="00812b9cdbc319496433de24ba35ff40bb40193855289fa4e4959a197fbc8321" Jul 02 14:19:01.919284 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.919266 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:19:01.989081 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:01.989044 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645c47d5c6-8br79"] Jul 02 14:19:02.888596 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:02.888516 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:02.920701 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:02.920663 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-4dnzn" event={"ID":"87a7e4f0-15dd-4ee3-bf66-9a0033fa496c","Type":"ContainerStarted","Data":"33de00d2681bc5429a04100e6a57a4a05e354e9c4353f8d92c48d7c95490e13d"} Jul 02 14:19:02.988238 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:02.988181 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:03.288339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:03.288239 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:03.388368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:03.388338 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:04.539411 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:04.539374 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:04.603167 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:04.603136 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:15.785525 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:15.785488 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:27.013337 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.013280 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-645c47d5c6-8br79" podUID="41945543-1d11-4e3b-9052-ce1836545a02" containerName="console" containerID="cri-o://a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3" gracePeriod=15 Jul 02 14:19:27.260488 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.260466 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:19:27.271316 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271269 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-oauth-serving-cert\") pod \"41945543-1d11-4e3b-9052-ce1836545a02\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " Jul 02 14:19:27.271407 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271319 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfb46\" (UniqueName: \"kubernetes.io/projected/41945543-1d11-4e3b-9052-ce1836545a02-kube-api-access-pfb46\") pod \"41945543-1d11-4e3b-9052-ce1836545a02\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " Jul 02 14:19:27.271514 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271496 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-oauth-config\") pod \"41945543-1d11-4e3b-9052-ce1836545a02\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " Jul 02 14:19:27.271577 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271542 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-console-config\") pod \"41945543-1d11-4e3b-9052-ce1836545a02\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " Jul 02 14:19:27.271630 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271606 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-serving-cert\") pod \"41945543-1d11-4e3b-9052-ce1836545a02\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " Jul 02 14:19:27.271679 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271644 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-service-ca\") pod \"41945543-1d11-4e3b-9052-ce1836545a02\" (UID: \"41945543-1d11-4e3b-9052-ce1836545a02\") " Jul 02 14:19:27.271727 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271674 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "41945543-1d11-4e3b-9052-ce1836545a02" (UID: "41945543-1d11-4e3b-9052-ce1836545a02"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:27.271858 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271845 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-oauth-serving-cert\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:19:27.272076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.271965 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-console-config" (OuterVolumeSpecName: "console-config") pod "41945543-1d11-4e3b-9052-ce1836545a02" (UID: "41945543-1d11-4e3b-9052-ce1836545a02"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:27.272219 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.272125 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-service-ca" (OuterVolumeSpecName: "service-ca") pod "41945543-1d11-4e3b-9052-ce1836545a02" (UID: "41945543-1d11-4e3b-9052-ce1836545a02"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:19:27.273618 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.273590 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "41945543-1d11-4e3b-9052-ce1836545a02" (UID: "41945543-1d11-4e3b-9052-ce1836545a02"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:19:27.273618 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.273602 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41945543-1d11-4e3b-9052-ce1836545a02-kube-api-access-pfb46" (OuterVolumeSpecName: "kube-api-access-pfb46") pod "41945543-1d11-4e3b-9052-ce1836545a02" (UID: "41945543-1d11-4e3b-9052-ce1836545a02"). InnerVolumeSpecName "kube-api-access-pfb46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:19:27.273618 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.273610 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "41945543-1d11-4e3b-9052-ce1836545a02" (UID: "41945543-1d11-4e3b-9052-ce1836545a02"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:19:27.372934 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.372891 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfb46\" (UniqueName: \"kubernetes.io/projected/41945543-1d11-4e3b-9052-ce1836545a02-kube-api-access-pfb46\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:19:27.372934 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.372922 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-oauth-config\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:19:27.372934 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.372932 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-console-config\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:19:27.372934 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.372943 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41945543-1d11-4e3b-9052-ce1836545a02-console-serving-cert\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:19:27.373317 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:27.372952 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41945543-1d11-4e3b-9052-ce1836545a02-service-ca\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:19:28.011575 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.011494 2581 generic.go:356] "Generic (PLEG): container finished" podID="41945543-1d11-4e3b-9052-ce1836545a02" containerID="a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3" exitCode=0 Jul 02 14:19:28.011575 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.011537 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-645c47d5c6-8br79" Jul 02 14:19:28.011775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.011582 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c47d5c6-8br79" event={"ID":"41945543-1d11-4e3b-9052-ce1836545a02","Type":"ContainerDied","Data":"a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3"} Jul 02 14:19:28.011775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.011632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-645c47d5c6-8br79" event={"ID":"41945543-1d11-4e3b-9052-ce1836545a02","Type":"ContainerDied","Data":"1f392730d449aad25c9e679f239674900557968b3f98bf7458a5fed3974bf975"} Jul 02 14:19:28.011775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.011657 2581 scope.go:117] "RemoveContainer" containerID="a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3" Jul 02 14:19:28.020230 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.020026 2581 scope.go:117] "RemoveContainer" containerID="a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3" Jul 02 14:19:28.020463 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:19:28.020325 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3\": container with ID starting with a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3 not found: ID does not exist" containerID="a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3" Jul 02 14:19:28.020463 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.020347 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3"} err="failed to get container status \"a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3\": rpc error: code = NotFound desc = could not find container \"a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3\": container with ID starting with a4e5eba1d732324f0689116853c7af4780abdbff32fc061865de62d2e84c73b3 not found: ID does not exist" Jul 02 14:19:28.036204 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.036177 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-645c47d5c6-8br79"] Jul 02 14:19:28.040882 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.040865 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-645c47d5c6-8br79"] Jul 02 14:19:28.284510 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:28.284433 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41945543-1d11-4e3b-9052-ce1836545a02" path="/var/lib/kubelet/pods/41945543-1d11-4e3b-9052-ce1836545a02/volumes" Jul 02 14:19:38.575297 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:38.575266 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:38.636464 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:38.636427 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:47.816356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.816306 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8d856778f-b48qn"] Jul 02 14:19:47.816780 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.816685 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41945543-1d11-4e3b-9052-ce1836545a02" containerName="console" Jul 02 14:19:47.816780 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.816700 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="41945543-1d11-4e3b-9052-ce1836545a02" containerName="console" Jul 02 14:19:47.816780 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.816758 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="41945543-1d11-4e3b-9052-ce1836545a02" containerName="console" Jul 02 14:19:47.819581 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.819564 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.831063 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.831038 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d856778f-b48qn"] Jul 02 14:19:47.884469 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.884438 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:47.933838 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.933802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-oauth-config\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.933838 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.933842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-service-ca\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.934101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.933915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-oauth-serving-cert\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.934101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.933965 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fprmr\" (UniqueName: \"kubernetes.io/projected/9d2dccbe-248c-4384-b251-929e27ebba55-kube-api-access-fprmr\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.934101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.934016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-console-config\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.934101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.934039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-trusted-ca-bundle\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.934101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.934092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-serving-cert\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:47.985579 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:47.985550 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:48.035071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-oauth-config\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035072 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-service-ca\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035277 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-oauth-serving-cert\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035277 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035118 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprmr\" (UniqueName: \"kubernetes.io/projected/9d2dccbe-248c-4384-b251-929e27ebba55-kube-api-access-fprmr\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035277 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-console-config\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035277 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035193 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-trusted-ca-bundle\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035277 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035217 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-serving-cert\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.035916 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.035891 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-service-ca\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.036025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.036001 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-console-config\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.036025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.036018 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-oauth-serving-cert\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.036350 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.036327 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-trusted-ca-bundle\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.037581 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.037554 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-oauth-config\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.037670 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.037654 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-serving-cert\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.044552 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.044533 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fprmr\" (UniqueName: \"kubernetes.io/projected/9d2dccbe-248c-4384-b251-929e27ebba55-kube-api-access-fprmr\") pod \"console-8d856778f-b48qn\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.128852 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.128824 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:48.251041 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:48.251015 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8d856778f-b48qn"] Jul 02 14:19:48.253365 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:19:48.253338 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2dccbe_248c_4384_b251_929e27ebba55.slice/crio-a5b424f17d139360245e0a43d90256c11395927fafa225f5fcc8dd3fc0d3d9f3 WatchSource:0}: Error finding container a5b424f17d139360245e0a43d90256c11395927fafa225f5fcc8dd3fc0d3d9f3: Status 404 returned error can't find the container with id a5b424f17d139360245e0a43d90256c11395927fafa225f5fcc8dd3fc0d3d9f3 Jul 02 14:19:49.093748 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:49.093714 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d856778f-b48qn" event={"ID":"9d2dccbe-248c-4384-b251-929e27ebba55","Type":"ContainerStarted","Data":"b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b"} Jul 02 14:19:49.094181 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:49.093762 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d856778f-b48qn" event={"ID":"9d2dccbe-248c-4384-b251-929e27ebba55","Type":"ContainerStarted","Data":"a5b424f17d139360245e0a43d90256c11395927fafa225f5fcc8dd3fc0d3d9f3"} Jul 02 14:19:49.114939 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:49.114892 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8d856778f-b48qn" podStartSLOduration=2.114876931 podStartE2EDuration="2.114876931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:19:49.11310836 +0000 UTC m=+193.234350600" watchObservedRunningTime="2026-07-02 14:19:49.114876931 +0000 UTC m=+193.236119244" Jul 02 14:19:50.367800 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:50.367766 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:50.437881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:50.437848 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:58.129431 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:58.129396 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:58.129820 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:58.129488 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:58.134302 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:58.134279 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:59.135311 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:59.135283 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:19:59.186273 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:59.186241 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75f66dbf88-7tjvf"] Jul 02 14:19:59.297565 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:59.297533 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:59.360783 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:59.360751 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:19:59.420337 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:19:59.420244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:04.544503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:04.544468 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:04.645593 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:04.645562 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:24.213857 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.213798 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-75f66dbf88-7tjvf" podUID="5a684867-c209-4781-94b1-af0f470b9ee6" containerName="console" containerID="cri-o://a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c" gracePeriod=15 Jul 02 14:20:24.452409 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.452384 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:20:24.518094 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518012 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-service-ca\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518094 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518055 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrfv4\" (UniqueName: \"kubernetes.io/projected/5a684867-c209-4781-94b1-af0f470b9ee6-kube-api-access-zrfv4\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518094 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518086 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-oauth-config\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518129 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-serving-cert\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518144 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-trusted-ca-bundle\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518192 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-oauth-serving-cert\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518368 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518218 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-console-config\") pod \"5a684867-c209-4781-94b1-af0f470b9ee6\" (UID: \"5a684867-c209-4781-94b1-af0f470b9ee6\") " Jul 02 14:20:24.518567 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518469 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:20:24.518687 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518658 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-console-config" (OuterVolumeSpecName: "console-config") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:20:24.518687 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518673 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:20:24.518777 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.518689 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:20:24.520378 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.520346 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:20:24.520483 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.520376 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:20:24.520483 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.520426 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a684867-c209-4781-94b1-af0f470b9ee6-kube-api-access-zrfv4" (OuterVolumeSpecName: "kube-api-access-zrfv4") pod "5a684867-c209-4781-94b1-af0f470b9ee6" (UID: "5a684867-c209-4781-94b1-af0f470b9ee6"). InnerVolumeSpecName "kube-api-access-zrfv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:20:24.619558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619519 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrfv4\" (UniqueName: \"kubernetes.io/projected/5a684867-c209-4781-94b1-af0f470b9ee6-kube-api-access-zrfv4\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:24.619558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619550 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-oauth-config\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:24.619558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619559 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a684867-c209-4781-94b1-af0f470b9ee6-console-serving-cert\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:24.619558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619567 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-trusted-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:24.619809 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619576 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-oauth-serving-cert\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:24.619809 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619584 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-console-config\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:24.619809 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:24.619604 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a684867-c209-4781-94b1-af0f470b9ee6-service-ca\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:20:25.220572 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.220532 2581 generic.go:356] "Generic (PLEG): container finished" podID="5a684867-c209-4781-94b1-af0f470b9ee6" containerID="a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c" exitCode=0 Jul 02 14:20:25.220983 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.220580 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75f66dbf88-7tjvf" Jul 02 14:20:25.220983 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.220581 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f66dbf88-7tjvf" event={"ID":"5a684867-c209-4781-94b1-af0f470b9ee6","Type":"ContainerDied","Data":"a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c"} Jul 02 14:20:25.220983 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.220634 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75f66dbf88-7tjvf" event={"ID":"5a684867-c209-4781-94b1-af0f470b9ee6","Type":"ContainerDied","Data":"6bc6da890304b21553ecd9789f535f90cb859be92be89aa96868b7bc978030ec"} Jul 02 14:20:25.220983 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.220661 2581 scope.go:117] "RemoveContainer" containerID="a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c" Jul 02 14:20:25.229210 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.229185 2581 scope.go:117] "RemoveContainer" containerID="a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c" Jul 02 14:20:25.229460 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:20:25.229438 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c\": container with ID starting with a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c not found: ID does not exist" containerID="a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c" Jul 02 14:20:25.229530 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.229471 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c"} err="failed to get container status \"a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c\": rpc error: code = NotFound desc = could not find container \"a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c\": container with ID starting with a213378e0d1fd6ff1a5fc00ff6eb447d8740f77e3ce33546d21859ccfcac914c not found: ID does not exist" Jul 02 14:20:25.245596 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.245571 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75f66dbf88-7tjvf"] Jul 02 14:20:25.251422 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:25.251398 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75f66dbf88-7tjvf"] Jul 02 14:20:26.285633 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:26.285601 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a684867-c209-4781-94b1-af0f470b9ee6" path="/var/lib/kubelet/pods/5a684867-c209-4781-94b1-af0f470b9ee6/volumes" Jul 02 14:20:27.862645 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:27.862610 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:27.964535 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:27.964500 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:28.082443 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:28.082408 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:55.490024 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:55.489992 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:55.647582 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:55.647536 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:55.772140 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:55.772040 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:56.044218 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:56.044108 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:56.147752 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:56.147724 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:56.263585 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:56.263556 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:20:57.684941 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.684908 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9"] Jul 02 14:20:57.685324 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.685171 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a684867-c209-4781-94b1-af0f470b9ee6" containerName="console" Jul 02 14:20:57.685324 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.685182 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a684867-c209-4781-94b1-af0f470b9ee6" containerName="console" Jul 02 14:20:57.685324 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.685229 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a684867-c209-4781-94b1-af0f470b9ee6" containerName="console" Jul 02 14:20:57.687029 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.686993 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.695379 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.695355 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wpbcz\"" Jul 02 14:20:57.695553 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.695537 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 14:20:57.696583 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.696569 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 14:20:57.718559 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.718534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9"] Jul 02 14:20:57.760785 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.760752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.760942 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.760823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbw6p\" (UniqueName: \"kubernetes.io/projected/9deae867-328d-4c27-abff-e117573e5506-kube-api-access-hbw6p\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.760942 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.760871 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.861499 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.861465 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.861682 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.861509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.861682 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.861551 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbw6p\" (UniqueName: \"kubernetes.io/projected/9deae867-328d-4c27-abff-e117573e5506-kube-api-access-hbw6p\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.861854 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.861833 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.861944 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.861922 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.873167 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.873136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbw6p\" (UniqueName: \"kubernetes.io/projected/9deae867-328d-4c27-abff-e117573e5506-kube-api-access-hbw6p\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:57.995586 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:57.995504 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:20:58.127076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:58.126980 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9"] Jul 02 14:20:58.129689 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:20:58.129657 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9deae867_328d_4c27_abff_e117573e5506.slice/crio-310661165d98377b812425048eb019065729cbab7e2c5c0630c324291dade0b3 WatchSource:0}: Error finding container 310661165d98377b812425048eb019065729cbab7e2c5c0630c324291dade0b3: Status 404 returned error can't find the container with id 310661165d98377b812425048eb019065729cbab7e2c5c0630c324291dade0b3 Jul 02 14:20:58.334447 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:20:58.334364 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" event={"ID":"9deae867-328d-4c27-abff-e117573e5506","Type":"ContainerStarted","Data":"310661165d98377b812425048eb019065729cbab7e2c5c0630c324291dade0b3"} Jul 02 14:21:06.365680 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:06.365638 2581 generic.go:356] "Generic (PLEG): container finished" podID="9deae867-328d-4c27-abff-e117573e5506" containerID="240563ffb285280470ace40e78b9de511cbb5338e0fc6ee190418a8fb2755325" exitCode=0 Jul 02 14:21:06.365680 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:06.365676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" event={"ID":"9deae867-328d-4c27-abff-e117573e5506","Type":"ContainerDied","Data":"240563ffb285280470ace40e78b9de511cbb5338e0fc6ee190418a8fb2755325"} Jul 02 14:21:07.779363 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:07.779329 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:09.378496 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:09.378462 2581 generic.go:356] "Generic (PLEG): container finished" podID="9deae867-328d-4c27-abff-e117573e5506" containerID="eaece171ce5287e828b0d3579d992ff0d45328e0e2c164e7eeacd9fff11b410a" exitCode=0 Jul 02 14:21:09.378890 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:09.378505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" event={"ID":"9deae867-328d-4c27-abff-e117573e5506","Type":"ContainerDied","Data":"eaece171ce5287e828b0d3579d992ff0d45328e0e2c164e7eeacd9fff11b410a"} Jul 02 14:21:18.417895 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:18.417868 2581 generic.go:356] "Generic (PLEG): container finished" podID="9deae867-328d-4c27-abff-e117573e5506" containerID="fc27bcde8bb55b2ee629d98322ee3431b81e8fa260433f49115ab1eae9be3960" exitCode=0 Jul 02 14:21:18.418223 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:18.417946 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" event={"ID":"9deae867-328d-4c27-abff-e117573e5506","Type":"ContainerDied","Data":"fc27bcde8bb55b2ee629d98322ee3431b81e8fa260433f49115ab1eae9be3960"} Jul 02 14:21:19.547197 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.547176 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:21:19.635912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.635883 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-bundle\") pod \"9deae867-328d-4c27-abff-e117573e5506\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " Jul 02 14:21:19.636065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.635969 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbw6p\" (UniqueName: \"kubernetes.io/projected/9deae867-328d-4c27-abff-e117573e5506-kube-api-access-hbw6p\") pod \"9deae867-328d-4c27-abff-e117573e5506\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " Jul 02 14:21:19.636065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.635999 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-util\") pod \"9deae867-328d-4c27-abff-e117573e5506\" (UID: \"9deae867-328d-4c27-abff-e117573e5506\") " Jul 02 14:21:19.636614 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.636584 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-bundle" (OuterVolumeSpecName: "bundle") pod "9deae867-328d-4c27-abff-e117573e5506" (UID: "9deae867-328d-4c27-abff-e117573e5506"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:21:19.638017 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.637989 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9deae867-328d-4c27-abff-e117573e5506-kube-api-access-hbw6p" (OuterVolumeSpecName: "kube-api-access-hbw6p") pod "9deae867-328d-4c27-abff-e117573e5506" (UID: "9deae867-328d-4c27-abff-e117573e5506"). InnerVolumeSpecName "kube-api-access-hbw6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:21:19.640439 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.640413 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-util" (OuterVolumeSpecName: "util") pod "9deae867-328d-4c27-abff-e117573e5506" (UID: "9deae867-328d-4c27-abff-e117573e5506"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 14:21:19.737435 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.737348 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:21:19.737435 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.737377 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbw6p\" (UniqueName: \"kubernetes.io/projected/9deae867-328d-4c27-abff-e117573e5506-kube-api-access-hbw6p\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:21:19.737435 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:19.737386 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9deae867-328d-4c27-abff-e117573e5506-util\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:21:20.427065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:20.427034 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" event={"ID":"9deae867-328d-4c27-abff-e117573e5506","Type":"ContainerDied","Data":"310661165d98377b812425048eb019065729cbab7e2c5c0630c324291dade0b3"} Jul 02 14:21:20.427065 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:20.427051 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mggn9" Jul 02 14:21:20.427282 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:20.427081 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="310661165d98377b812425048eb019065729cbab7e2c5c0630c324291dade0b3" Jul 02 14:21:24.700675 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:24.700642 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:24.857302 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:24.857266 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:26.914537 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914504 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs"] Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914803 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="extract" Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914814 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="extract" Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914831 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="pull" Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914836 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="pull" Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914842 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="util" Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914860 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="util" Jul 02 14:21:26.914971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.914902 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9deae867-328d-4c27-abff-e117573e5506" containerName="extract" Jul 02 14:21:26.917862 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.917845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:26.921173 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.921138 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-bqr8z\"" Jul 02 14:21:26.921400 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.921385 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 02 14:21:26.921597 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.921582 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 02 14:21:26.921794 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.921748 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 02 14:21:26.934851 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:26.934823 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs"] Jul 02 14:21:27.090069 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.090031 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th7gg\" (UniqueName: \"kubernetes.io/projected/20c7dd01-d5cd-4e59-b335-63683f47c828-kube-api-access-th7gg\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs\" (UID: \"20c7dd01-d5cd-4e59-b335-63683f47c828\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.090267 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.090076 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/20c7dd01-d5cd-4e59-b335-63683f47c828-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs\" (UID: \"20c7dd01-d5cd-4e59-b335-63683f47c828\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.190871 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.190773 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th7gg\" (UniqueName: \"kubernetes.io/projected/20c7dd01-d5cd-4e59-b335-63683f47c828-kube-api-access-th7gg\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs\" (UID: \"20c7dd01-d5cd-4e59-b335-63683f47c828\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.190871 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.190822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/20c7dd01-d5cd-4e59-b335-63683f47c828-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs\" (UID: \"20c7dd01-d5cd-4e59-b335-63683f47c828\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.193031 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.193005 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/20c7dd01-d5cd-4e59-b335-63683f47c828-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs\" (UID: \"20c7dd01-d5cd-4e59-b335-63683f47c828\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.203558 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.203531 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th7gg\" (UniqueName: \"kubernetes.io/projected/20c7dd01-d5cd-4e59-b335-63683f47c828-kube-api-access-th7gg\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs\" (UID: \"20c7dd01-d5cd-4e59-b335-63683f47c828\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.229353 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.229333 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:27.374259 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.374205 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs"] Jul 02 14:21:27.378372 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:21:27.378337 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c7dd01_d5cd_4e59_b335_63683f47c828.slice/crio-4cbd58b6c4b3e41244e1b3d0031d84cd796e99c898355f2d12aadc830b8a2d75 WatchSource:0}: Error finding container 4cbd58b6c4b3e41244e1b3d0031d84cd796e99c898355f2d12aadc830b8a2d75: Status 404 returned error can't find the container with id 4cbd58b6c4b3e41244e1b3d0031d84cd796e99c898355f2d12aadc830b8a2d75 Jul 02 14:21:27.456775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:27.456694 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" event={"ID":"20c7dd01-d5cd-4e59-b335-63683f47c828","Type":"ContainerStarted","Data":"4cbd58b6c4b3e41244e1b3d0031d84cd796e99c898355f2d12aadc830b8a2d75"} Jul 02 14:21:30.471246 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.471129 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" event={"ID":"20c7dd01-d5cd-4e59-b335-63683f47c828","Type":"ContainerStarted","Data":"313e8bc2b85449905a9a2d19a1208299f70069f65cf2a08797cebd4b70cefb59"} Jul 02 14:21:30.471717 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.471271 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:30.493453 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.493403 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" podStartSLOduration=1.67612805 podStartE2EDuration="4.493387403s" totalImagesPullingTime="2.817259353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:21:30.492432049 +0000 UTC m=+294.613674299" watchObservedRunningTime="2026-07-02 14:21:30.493387403 +0000 UTC m=+294.614629642" Jul 02 14:21:30.629344 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.629310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:30.953663 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.953624 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:30.965818 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.965789 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-96kd4"] Jul 02 14:21:30.967921 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.967906 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:30.970594 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.970575 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 02 14:21:30.970708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.970665 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 02 14:21:30.970708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.970671 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-8cfr9\"" Jul 02 14:21:30.979438 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:30.979417 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-96kd4"] Jul 02 14:21:31.075857 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.075817 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:31.122091 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.122056 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.122257 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.122113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-cabundle0\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.122257 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.122184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m59h\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-kube-api-access-9m59h\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.223506 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.223414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-cabundle0\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.223506 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.223469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m59h\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-kube-api-access-9m59h\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.223735 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.223508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.223735 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.223604 2581 secret.go:281] references non-existent secret key: ca.crt Jul 02 14:21:31.223735 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.223618 2581 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 02 14:21:31.223735 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.223626 2581 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-96kd4: references non-existent secret key: ca.crt Jul 02 14:21:31.223735 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.223692 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates podName:7cca4870-1194-4b73-86e9-bf4e0c7cd93a nodeName:}" failed. No retries permitted until 2026-07-02 14:21:31.723671444 +0000 UTC m=+295.844913679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates") pod "keda-operator-68bff59c-96kd4" (UID: "7cca4870-1194-4b73-86e9-bf4e0c7cd93a") : references non-existent secret key: ca.crt Jul 02 14:21:31.224078 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.224057 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-cabundle0\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.233066 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.233038 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m59h\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-kube-api-access-9m59h\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.605693 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.605604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:31.727869 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.727825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:31.728039 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.727966 2581 secret.go:281] references non-existent secret key: ca.crt Jul 02 14:21:31.728039 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.727985 2581 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 02 14:21:31.728039 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.727995 2581 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-96kd4: references non-existent secret key: ca.crt Jul 02 14:21:31.728147 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:21:31.728057 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates podName:7cca4870-1194-4b73-86e9-bf4e0c7cd93a nodeName:}" failed. No retries permitted until 2026-07-02 14:21:32.728041261 +0000 UTC m=+296.849283479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates") pod "keda-operator-68bff59c-96kd4" (UID: "7cca4870-1194-4b73-86e9-bf4e0c7cd93a") : references non-existent secret key: ca.crt Jul 02 14:21:31.960956 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:31.960920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:32.248810 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.248723 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:32.278611 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.278581 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-tsc5w"] Jul 02 14:21:32.281134 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.281112 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.284939 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.284919 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 02 14:21:32.293271 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.293248 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-tsc5w"] Jul 02 14:21:32.360846 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.360813 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:32.434147 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.434108 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/aa7ccd66-9aec-4882-bf55-2e074a3bb783-certificates\") pod \"keda-admission-5fbd5c4644-tsc5w\" (UID: \"aa7ccd66-9aec-4882-bf55-2e074a3bb783\") " pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.434361 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.434255 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h8b2\" (UniqueName: \"kubernetes.io/projected/aa7ccd66-9aec-4882-bf55-2e074a3bb783-kube-api-access-7h8b2\") pod \"keda-admission-5fbd5c4644-tsc5w\" (UID: \"aa7ccd66-9aec-4882-bf55-2e074a3bb783\") " pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.476100 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.476056 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:32.535530 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.535443 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h8b2\" (UniqueName: \"kubernetes.io/projected/aa7ccd66-9aec-4882-bf55-2e074a3bb783-kube-api-access-7h8b2\") pod \"keda-admission-5fbd5c4644-tsc5w\" (UID: \"aa7ccd66-9aec-4882-bf55-2e074a3bb783\") " pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.535530 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.535499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/aa7ccd66-9aec-4882-bf55-2e074a3bb783-certificates\") pod \"keda-admission-5fbd5c4644-tsc5w\" (UID: \"aa7ccd66-9aec-4882-bf55-2e074a3bb783\") " pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.537976 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.537952 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/aa7ccd66-9aec-4882-bf55-2e074a3bb783-certificates\") pod \"keda-admission-5fbd5c4644-tsc5w\" (UID: \"aa7ccd66-9aec-4882-bf55-2e074a3bb783\") " pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.546862 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.546838 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h8b2\" (UniqueName: \"kubernetes.io/projected/aa7ccd66-9aec-4882-bf55-2e074a3bb783-kube-api-access-7h8b2\") pod \"keda-admission-5fbd5c4644-tsc5w\" (UID: \"aa7ccd66-9aec-4882-bf55-2e074a3bb783\") " pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.591712 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.591686 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:32.675342 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.675309 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:32.724867 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.724840 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-tsc5w"] Jul 02 14:21:32.727354 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:21:32.727331 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7ccd66_9aec_4882_bf55_2e074a3bb783.slice/crio-8b101820a3c70dea93db4ee2661f39be84b1c025f73059cfee889ae38f0390ae WatchSource:0}: Error finding container 8b101820a3c70dea93db4ee2661f39be84b1c025f73059cfee889ae38f0390ae: Status 404 returned error can't find the container with id 8b101820a3c70dea93db4ee2661f39be84b1c025f73059cfee889ae38f0390ae Jul 02 14:21:32.737047 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.737027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:32.739543 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.739517 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7cca4870-1194-4b73-86e9-bf4e0c7cd93a-certificates\") pod \"keda-operator-68bff59c-96kd4\" (UID: \"7cca4870-1194-4b73-86e9-bf4e0c7cd93a\") " pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:32.777656 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.777622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:32.828406 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.828374 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:32.904422 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:32.904391 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-96kd4"] Jul 02 14:21:32.907051 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:21:32.907022 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cca4870_1194_4b73_86e9_bf4e0c7cd93a.slice/crio-852b2f94031f33bb071cff4e9aef82cdfd6eecd0988aaed0bfeb219f527c4e5b WatchSource:0}: Error finding container 852b2f94031f33bb071cff4e9aef82cdfd6eecd0988aaed0bfeb219f527c4e5b: Status 404 returned error can't find the container with id 852b2f94031f33bb071cff4e9aef82cdfd6eecd0988aaed0bfeb219f527c4e5b Jul 02 14:21:33.055197 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:33.055078 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:33.159418 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:33.159387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:21:33.483910 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:33.483874 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-96kd4" event={"ID":"7cca4870-1194-4b73-86e9-bf4e0c7cd93a","Type":"ContainerStarted","Data":"852b2f94031f33bb071cff4e9aef82cdfd6eecd0988aaed0bfeb219f527c4e5b"} Jul 02 14:21:33.484758 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:33.484726 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" event={"ID":"aa7ccd66-9aec-4882-bf55-2e074a3bb783","Type":"ContainerStarted","Data":"8b101820a3c70dea93db4ee2661f39be84b1c025f73059cfee889ae38f0390ae"} Jul 02 14:21:36.229396 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.229371 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:21:36.229827 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.229417 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:21:36.239936 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.239916 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 14:21:36.500386 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.500349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-96kd4" event={"ID":"7cca4870-1194-4b73-86e9-bf4e0c7cd93a","Type":"ContainerStarted","Data":"641bcd092954f4283975abf9724f263a2bb0dc324401e9885d36e78a10dcf8b2"} Jul 02 14:21:36.501836 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.500415 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:36.501836 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.501745 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" event={"ID":"aa7ccd66-9aec-4882-bf55-2e074a3bb783","Type":"ContainerStarted","Data":"3c1f89b512aaf49a2058ae82b0c7d0df14918e92509affb0ca977d68ee106b78"} Jul 02 14:21:36.501836 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.501805 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:21:36.523741 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.523690 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-96kd4" podStartSLOduration=3.334451432 podStartE2EDuration="6.523675842s" totalImagesPullingTime="3.18922441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:21:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:21:36.522099299 +0000 UTC m=+300.643341540" watchObservedRunningTime="2026-07-02 14:21:36.523675842 +0000 UTC m=+300.644918088" Jul 02 14:21:36.541124 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:36.541081 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" podStartSLOduration=2.236641943 podStartE2EDuration="4.541066776s" totalImagesPullingTime="2.304424833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:21:36.540042426 +0000 UTC m=+300.661284667" watchObservedRunningTime="2026-07-02 14:21:36.541066776 +0000 UTC m=+300.662309048" Jul 02 14:21:51.478234 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:51.478204 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-zmnjs" Jul 02 14:21:57.509095 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:57.509062 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-96kd4" Jul 02 14:21:57.509606 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:21:57.509138 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-tsc5w" Jul 02 14:22:08.397321 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:08.397286 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:10.400917 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:10.400873 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:38.361123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.361042 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:38.433277 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.433245 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:38.444413 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.444384 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-tcdjh"] Jul 02 14:22:38.446880 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.446862 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.449630 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.449604 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 02 14:22:38.449744 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.449642 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 02 14:22:38.450674 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.450660 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-x8wl7\"" Jul 02 14:22:38.450731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.450680 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 02 14:22:38.458840 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.458819 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-tcdjh"] Jul 02 14:22:38.532551 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.532519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx4r5\" (UniqueName: \"kubernetes.io/projected/e3869f4f-1b7b-44fc-b165-1dfcb925e876-kube-api-access-nx4r5\") pod \"llmisvc-controller-manager-67d95fdc48-tcdjh\" (UID: \"e3869f4f-1b7b-44fc-b165-1dfcb925e876\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.532718 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.532574 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3869f4f-1b7b-44fc-b165-1dfcb925e876-cert\") pod \"llmisvc-controller-manager-67d95fdc48-tcdjh\" (UID: \"e3869f4f-1b7b-44fc-b165-1dfcb925e876\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.536665 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.536641 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:38.632994 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.632958 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx4r5\" (UniqueName: \"kubernetes.io/projected/e3869f4f-1b7b-44fc-b165-1dfcb925e876-kube-api-access-nx4r5\") pod \"llmisvc-controller-manager-67d95fdc48-tcdjh\" (UID: \"e3869f4f-1b7b-44fc-b165-1dfcb925e876\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.633175 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.633018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3869f4f-1b7b-44fc-b165-1dfcb925e876-cert\") pod \"llmisvc-controller-manager-67d95fdc48-tcdjh\" (UID: \"e3869f4f-1b7b-44fc-b165-1dfcb925e876\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.635493 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.635470 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3869f4f-1b7b-44fc-b165-1dfcb925e876-cert\") pod \"llmisvc-controller-manager-67d95fdc48-tcdjh\" (UID: \"e3869f4f-1b7b-44fc-b165-1dfcb925e876\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.635912 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.635888 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:38.643220 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.643191 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx4r5\" (UniqueName: \"kubernetes.io/projected/e3869f4f-1b7b-44fc-b165-1dfcb925e876-kube-api-access-nx4r5\") pod \"llmisvc-controller-manager-67d95fdc48-tcdjh\" (UID: \"e3869f4f-1b7b-44fc-b165-1dfcb925e876\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.757199 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.757171 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:38.890275 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.890250 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-tcdjh"] Jul 02 14:22:38.892434 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:22:38.892391 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode3869f4f_1b7b_44fc_b165_1dfcb925e876.slice/crio-92d71a88e2a70b1b7e7c05460fc75fb7720bb28c434adb169b20fe072431def3 WatchSource:0}: Error finding container 92d71a88e2a70b1b7e7c05460fc75fb7720bb28c434adb169b20fe072431def3: Status 404 returned error can't find the container with id 92d71a88e2a70b1b7e7c05460fc75fb7720bb28c434adb169b20fe072431def3 Jul 02 14:22:38.894124 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.894110 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 14:22:38.970812 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:38.970779 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:39.071047 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:39.071011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:39.171067 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:39.170976 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:22:39.762133 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:39.762097 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" event={"ID":"e3869f4f-1b7b-44fc-b165-1dfcb925e876","Type":"ContainerStarted","Data":"92d71a88e2a70b1b7e7c05460fc75fb7720bb28c434adb169b20fe072431def3"} Jul 02 14:22:40.767099 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:40.767074 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" event={"ID":"e3869f4f-1b7b-44fc-b165-1dfcb925e876","Type":"ContainerStarted","Data":"2c6677004337758f062aaceaab6034ea8877ff5e6f1286126f53bad5f409d3a5"} Jul 02 14:22:40.767421 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:40.767190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:22:40.788962 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:22:40.788918 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" podStartSLOduration=0.976999498 podStartE2EDuration="2.788903261s" totalImagesPullingTime="1.811903763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:22:40.787412469 +0000 UTC m=+364.908654711" watchObservedRunningTime="2026-07-02 14:22:40.788903261 +0000 UTC m=+364.910145501" Jul 02 14:23:05.314071 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:05.314037 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:05.386038 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:05.386004 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:07.365084 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:07.365050 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:07.468244 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:07.468206 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:11.773971 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:11.773943 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-tcdjh" Jul 02 14:23:27.756077 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:27.756040 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:27.813569 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:27.813535 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:47.333945 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:47.333906 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:47.418085 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:47.418054 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:47.487440 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:47.487410 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:47.790446 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:47.790412 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:48.100799 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:48.100715 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:48.161243 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:48.161208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:52.867012 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:52.866978 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75458599c4-jtvql"] Jul 02 14:23:52.869994 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:52.869972 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:52.884096 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:52.884068 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75458599c4-jtvql"] Jul 02 14:23:52.950616 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:52.950590 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:53.020785 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.020749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-trusted-ca-bundle\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.020966 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.020797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-oauth-config\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.020966 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.020872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-serving-cert\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.020966 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.020907 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-oauth-serving-cert\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.020966 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.020933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-config\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.021118 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.020973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-service-ca\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.021118 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.021051 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mjph\" (UniqueName: \"kubernetes.io/projected/faf7de05-72ac-4b13-8049-18d5c7fc892c-kube-api-access-4mjph\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.048913 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.048885 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:53.122104 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122023 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mjph\" (UniqueName: \"kubernetes.io/projected/faf7de05-72ac-4b13-8049-18d5c7fc892c-kube-api-access-4mjph\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.122104 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-trusted-ca-bundle\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.122104 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-oauth-config\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.122425 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-serving-cert\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.122425 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122149 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-oauth-serving-cert\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.122425 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122196 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-config\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.122425 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122218 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-service-ca\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.123012 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.122973 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-oauth-serving-cert\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.123130 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.123023 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-trusted-ca-bundle\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.123301 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.123282 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-config\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.123375 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.123342 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/faf7de05-72ac-4b13-8049-18d5c7fc892c-service-ca\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.124637 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.124620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-oauth-config\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.124731 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.124717 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/faf7de05-72ac-4b13-8049-18d5c7fc892c-console-serving-cert\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.132202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.132184 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mjph\" (UniqueName: \"kubernetes.io/projected/faf7de05-72ac-4b13-8049-18d5c7fc892c-kube-api-access-4mjph\") pod \"console-75458599c4-jtvql\" (UID: \"faf7de05-72ac-4b13-8049-18d5c7fc892c\") " pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.150689 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.150665 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:53.179880 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.179858 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:23:53.315886 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.315859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75458599c4-jtvql"] Jul 02 14:23:53.317905 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:23:53.317880 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf7de05_72ac_4b13_8049_18d5c7fc892c.slice/crio-e3e555e93ff4703c6171f5d95841b13995cd1b1d1c0c4f6f25e5c264b3997f0a WatchSource:0}: Error finding container e3e555e93ff4703c6171f5d95841b13995cd1b1d1c0c4f6f25e5c264b3997f0a: Status 404 returned error can't find the container with id e3e555e93ff4703c6171f5d95841b13995cd1b1d1c0c4f6f25e5c264b3997f0a Jul 02 14:23:53.655622 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:53.655589 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:23:54.064129 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:54.064032 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75458599c4-jtvql" event={"ID":"faf7de05-72ac-4b13-8049-18d5c7fc892c","Type":"ContainerStarted","Data":"b0362f848c1d8896b8d423ecb40d6cf843e20c24344b13b03893b66980e7ccbc"} Jul 02 14:23:54.064129 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:54.064083 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75458599c4-jtvql" event={"ID":"faf7de05-72ac-4b13-8049-18d5c7fc892c","Type":"ContainerStarted","Data":"e3e555e93ff4703c6171f5d95841b13995cd1b1d1c0c4f6f25e5c264b3997f0a"} Jul 02 14:23:54.082544 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:23:54.082494 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75458599c4-jtvql" podStartSLOduration=2.082477085 podStartE2EDuration="2.082477085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:23:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:23:54.081275732 +0000 UTC m=+438.202517974" watchObservedRunningTime="2026-07-02 14:23:54.082477085 +0000 UTC m=+438.203719324" Jul 02 14:24:03.180529 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:03.180441 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:24:03.180529 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:03.180506 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:24:03.185447 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:03.185427 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:24:04.110584 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:04.110553 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75458599c4-jtvql" Jul 02 14:24:04.164210 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:04.164178 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8d856778f-b48qn"] Jul 02 14:24:04.337296 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:04.337266 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:24:04.416409 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:04.416378 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:24:04.506123 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:04.506089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:24:29.184730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.184668 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8d856778f-b48qn" podUID="9d2dccbe-248c-4384-b251-929e27ebba55" containerName="console" containerID="cri-o://b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b" gracePeriod=15 Jul 02 14:24:29.430456 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.430432 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:24:29.513557 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513477 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-serving-cert\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.513557 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513508 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-service-ca\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.513557 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513533 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-oauth-config\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.513557 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513562 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-trusted-ca-bundle\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.513873 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513583 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-oauth-serving-cert\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.513873 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513640 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fprmr\" (UniqueName: \"kubernetes.io/projected/9d2dccbe-248c-4384-b251-929e27ebba55-kube-api-access-fprmr\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.513873 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513661 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-console-config\") pod \"9d2dccbe-248c-4384-b251-929e27ebba55\" (UID: \"9d2dccbe-248c-4384-b251-929e27ebba55\") " Jul 02 14:24:29.514029 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.513950 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-service-ca" (OuterVolumeSpecName: "service-ca") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:24:29.514086 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.514068 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:24:29.514143 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.514081 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:24:29.514143 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.514117 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-console-config" (OuterVolumeSpecName: "console-config") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:24:29.515708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.515679 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2dccbe-248c-4384-b251-929e27ebba55-kube-api-access-fprmr" (OuterVolumeSpecName: "kube-api-access-fprmr") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "kube-api-access-fprmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 14:24:29.515816 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.515702 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:24:29.515816 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.515719 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9d2dccbe-248c-4384-b251-929e27ebba55" (UID: "9d2dccbe-248c-4384-b251-929e27ebba55"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:24:29.614551 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614516 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-serving-cert\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:29.614551 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614547 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-service-ca\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:29.614551 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614556 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d2dccbe-248c-4384-b251-929e27ebba55-console-oauth-config\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:29.614779 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614564 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-trusted-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:29.614779 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614572 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-oauth-serving-cert\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:29.614779 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614582 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fprmr\" (UniqueName: \"kubernetes.io/projected/9d2dccbe-248c-4384-b251-929e27ebba55-kube-api-access-fprmr\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:29.614779 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:29.614593 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d2dccbe-248c-4384-b251-929e27ebba55-console-config\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:24:30.208119 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.208087 2581 generic.go:356] "Generic (PLEG): container finished" podID="9d2dccbe-248c-4384-b251-929e27ebba55" containerID="b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b" exitCode=0 Jul 02 14:24:30.208529 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.208140 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8d856778f-b48qn" Jul 02 14:24:30.208529 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.208180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d856778f-b48qn" event={"ID":"9d2dccbe-248c-4384-b251-929e27ebba55","Type":"ContainerDied","Data":"b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b"} Jul 02 14:24:30.208529 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.208253 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8d856778f-b48qn" event={"ID":"9d2dccbe-248c-4384-b251-929e27ebba55","Type":"ContainerDied","Data":"a5b424f17d139360245e0a43d90256c11395927fafa225f5fcc8dd3fc0d3d9f3"} Jul 02 14:24:30.208529 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.208274 2581 scope.go:117] "RemoveContainer" containerID="b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b" Jul 02 14:24:30.216482 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.216466 2581 scope.go:117] "RemoveContainer" containerID="b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b" Jul 02 14:24:30.216744 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:24:30.216721 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b\": container with ID starting with b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b not found: ID does not exist" containerID="b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b" Jul 02 14:24:30.216813 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.216751 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b"} err="failed to get container status \"b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b\": rpc error: code = NotFound desc = could not find container \"b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b\": container with ID starting with b838c120296a9a3642a8732accbfcf94364f6ca3216c8784a7a061ebf9fc821b not found: ID does not exist" Jul 02 14:24:30.230526 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.230504 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8d856778f-b48qn"] Jul 02 14:24:30.234674 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.234654 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8d856778f-b48qn"] Jul 02 14:24:30.285113 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:24:30.285080 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2dccbe-248c-4384-b251-929e27ebba55" path="/var/lib/kubelet/pods/9d2dccbe-248c-4384-b251-929e27ebba55/volumes" Jul 02 14:25:23.546934 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:23.546887 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:23.643851 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:23.643806 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:34.556181 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:34.556074 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:35.103584 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:35.103539 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:35.332350 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:35.332319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:35.495238 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:35.495199 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:35.592763 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:35.592730 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:35.659504 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:35.659473 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:35.809550 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:35.809466 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:36.005716 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:36.005675 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:25:36.106977 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:25:36.106897 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:26:36.256594 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:36.256564 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:26:36.259203 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:36.259181 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:26:53.969984 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:53.969945 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:26:55.120222 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.120191 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:26:55.126614 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.126590 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d"] Jul 02 14:26:55.126872 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.126861 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2dccbe-248c-4384-b251-929e27ebba55" containerName="console" Jul 02 14:26:55.126914 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.126874 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2dccbe-248c-4384-b251-929e27ebba55" containerName="console" Jul 02 14:26:55.126948 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.126943 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2dccbe-248c-4384-b251-929e27ebba55" containerName="console" Jul 02 14:26:55.128644 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.128629 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.131179 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.131135 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-df2c1-kube-rbac-proxy-sar-config\"" Jul 02 14:26:55.131290 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.131237 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 02 14:26:55.131290 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.131246 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fn24b\"" Jul 02 14:26:55.131396 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.131317 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-df2c1-serving-cert\"" Jul 02 14:26:55.137824 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.137805 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d"] Jul 02 14:26:55.178895 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.178865 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-openshift-service-ca-bundle\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.179036 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.178903 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.279941 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.279898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-openshift-service-ca-bundle\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.279941 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.279944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.280199 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:26:55.280063 2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-df2c1-serving-cert: secret "switch-graph-df2c1-serving-cert" not found Jul 02 14:26:55.280199 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:26:55.280124 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls podName:66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1 nodeName:}" failed. No retries permitted until 2026-07-02 14:26:55.780100376 +0000 UTC m=+619.901342594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls") pod "switch-graph-df2c1-84fd8ddb75-snq4d" (UID: "66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1") : secret "switch-graph-df2c1-serving-cert" not found Jul 02 14:26:55.280565 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.280547 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-openshift-service-ca-bundle\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.785174 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.785127 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:55.785359 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:26:55.785276 2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-df2c1-serving-cert: secret "switch-graph-df2c1-serving-cert" not found Jul 02 14:26:55.785359 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:26:55.785339 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls podName:66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1 nodeName:}" failed. No retries permitted until 2026-07-02 14:26:56.785321814 +0000 UTC m=+620.906564039 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls") pod "switch-graph-df2c1-84fd8ddb75-snq4d" (UID: "66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1") : secret "switch-graph-df2c1-serving-cert" not found Jul 02 14:26:55.881127 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:55.881092 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:26:56.794480 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:56.794445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:56.796770 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:56.796750 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") pod \"switch-graph-df2c1-84fd8ddb75-snq4d\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:56.939114 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:56.939084 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:26:57.063119 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:57.063044 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d"] Jul 02 14:26:57.066008 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:26:57.065973 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66798a0f_7d3f_4bfa_a9c9_c7fac5fa54e1.slice/crio-1ae0eb561fdb16a164e386b4bcf921c8aa5f2bec664b6a48379943ba58767e27 WatchSource:0}: Error finding container 1ae0eb561fdb16a164e386b4bcf921c8aa5f2bec664b6a48379943ba58767e27: Status 404 returned error can't find the container with id 1ae0eb561fdb16a164e386b4bcf921c8aa5f2bec664b6a48379943ba58767e27 Jul 02 14:26:57.791524 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:57.791436 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" event={"ID":"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1","Type":"ContainerStarted","Data":"1ae0eb561fdb16a164e386b4bcf921c8aa5f2bec664b6a48379943ba58767e27"} Jul 02 14:26:59.606083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:26:59.606050 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:00.806366 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:00.806332 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" event={"ID":"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1","Type":"ContainerStarted","Data":"44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e"} Jul 02 14:27:00.806767 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:00.806460 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:27:06.817072 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:06.817038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:27:06.836309 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:06.836255 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podStartSLOduration=8.996090891 podStartE2EDuration="11.836237496s" totalImagesPullingTime="2.840146605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:27:00.830795624 +0000 UTC m=+624.952037878" watchObservedRunningTime="2026-07-02 14:27:06.836237496 +0000 UTC m=+630.957479735" Jul 02 14:27:09.255168 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.255122 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d"] Jul 02 14:27:09.255623 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.255418 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" containerID="cri-o://44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e" gracePeriod=30 Jul 02 14:27:09.280700 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.280662 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:09.432814 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.432751 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:09.508680 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.508609 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:09.592890 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.592859 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:09.689763 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.689733 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:09.790520 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:09.790420 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:11.815345 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:11.815303 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:27:16.817331 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:16.817287 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:27:21.814918 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:21.814874 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:27:21.815309 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:21.814990 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:27:26.815280 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:26.815239 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:27:31.814795 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:31.814753 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:27:36.814989 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:36.814941 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:27:39.902589 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.902564 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:27:39.948222 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.948190 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-openshift-service-ca-bundle\") pod \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " Jul 02 14:27:39.948365 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.948239 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") pod \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\" (UID: \"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1\") " Jul 02 14:27:39.948545 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.948524 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" (UID: "66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:27:39.950294 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.950270 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" (UID: "66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:27:39.976083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.976053 2581 generic.go:356] "Generic (PLEG): container finished" podID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerID="44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e" exitCode=0 Jul 02 14:27:39.976229 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.976092 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" Jul 02 14:27:39.976229 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.976131 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" event={"ID":"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1","Type":"ContainerDied","Data":"44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e"} Jul 02 14:27:39.976229 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.976201 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d" event={"ID":"66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1","Type":"ContainerDied","Data":"1ae0eb561fdb16a164e386b4bcf921c8aa5f2bec664b6a48379943ba58767e27"} Jul 02 14:27:39.976229 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.976226 2581 scope.go:117] "RemoveContainer" containerID="44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e" Jul 02 14:27:39.984580 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.984565 2581 scope.go:117] "RemoveContainer" containerID="44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e" Jul 02 14:27:39.984856 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:27:39.984840 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e\": container with ID starting with 44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e not found: ID does not exist" containerID="44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e" Jul 02 14:27:39.984908 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.984864 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e"} err="failed to get container status \"44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e\": rpc error: code = NotFound desc = could not find container \"44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e\": container with ID starting with 44939388c9a4551c78a0dc3a47177f841caa3c22cd6c2ca5d18721d5b6ac532e not found: ID does not exist" Jul 02 14:27:39.998341 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:39.998315 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d"] Jul 02 14:27:40.005777 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:40.005751 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-df2c1-84fd8ddb75-snq4d"] Jul 02 14:27:40.048901 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:40.048839 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:27:40.048901 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:40.048862 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:27:40.284754 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:40.284720 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" path="/var/lib/kubelet/pods/66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1/volumes" Jul 02 14:27:45.549732 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:45.549700 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:55.105828 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.105793 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s"] Jul 02 14:27:55.106222 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.106096 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" Jul 02 14:27:55.106222 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.106107 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" Jul 02 14:27:55.106222 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.106183 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="66798a0f-7d3f-4bfa-a9c9-c7fac5fa54e1" containerName="switch-graph-df2c1" Jul 02 14:27:55.109244 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.109228 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.113628 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.113598 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 02 14:27:55.113775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.113598 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Jul 02 14:27:55.113775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.113679 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Jul 02 14:27:55.113775 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.113740 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fn24b\"" Jul 02 14:27:55.116272 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.116245 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:55.122676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.122657 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s"] Jul 02 14:27:55.157525 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.157497 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d80103d8-904b-4db7-8756-141cc8462374-openshift-service-ca-bundle\") pod \"model-chainer-7d6d5bb5c4-kqn2s\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.157667 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.157573 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d80103d8-904b-4db7-8756-141cc8462374-proxy-tls\") pod \"model-chainer-7d6d5bb5c4-kqn2s\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.257938 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.257903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d80103d8-904b-4db7-8756-141cc8462374-proxy-tls\") pod \"model-chainer-7d6d5bb5c4-kqn2s\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.258105 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.257955 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d80103d8-904b-4db7-8756-141cc8462374-openshift-service-ca-bundle\") pod \"model-chainer-7d6d5bb5c4-kqn2s\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.258628 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.258610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d80103d8-904b-4db7-8756-141cc8462374-openshift-service-ca-bundle\") pod \"model-chainer-7d6d5bb5c4-kqn2s\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.260302 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.260286 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d80103d8-904b-4db7-8756-141cc8462374-proxy-tls\") pod \"model-chainer-7d6d5bb5c4-kqn2s\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.313695 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.313648 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:27:55.419582 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.419554 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:55.554970 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:55.554945 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s"] Jul 02 14:27:55.557653 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:27:55.557626 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80103d8_904b_4db7_8756_141cc8462374.slice/crio-c40cd1a843418ffda65e93a30a3485f6175da0479f3bbed1f14214fa9e5d7354 WatchSource:0}: Error finding container c40cd1a843418ffda65e93a30a3485f6175da0479f3bbed1f14214fa9e5d7354: Status 404 returned error can't find the container with id c40cd1a843418ffda65e93a30a3485f6175da0479f3bbed1f14214fa9e5d7354 Jul 02 14:27:56.041980 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:56.041943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" event={"ID":"d80103d8-904b-4db7-8756-141cc8462374","Type":"ContainerStarted","Data":"1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba"} Jul 02 14:27:56.042174 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:56.041998 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" event={"ID":"d80103d8-904b-4db7-8756-141cc8462374","Type":"ContainerStarted","Data":"c40cd1a843418ffda65e93a30a3485f6175da0479f3bbed1f14214fa9e5d7354"} Jul 02 14:27:56.042174 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:56.042034 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:27:56.060449 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:27:56.060376 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podStartSLOduration=1.060360948 podStartE2EDuration="1.060360948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:27:56.059383628 +0000 UTC m=+680.180625869" watchObservedRunningTime="2026-07-02 14:27:56.060360948 +0000 UTC m=+680.181603187" Jul 02 14:28:02.052162 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:02.052133 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:28:05.186876 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.186847 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s"] Jul 02 14:28:05.187278 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.187028 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" containerID="cri-o://1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba" gracePeriod=30 Jul 02 14:28:05.225539 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.225508 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:05.326750 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.326701 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:05.430027 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.429998 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:05.530782 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.530707 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:05.641994 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.641962 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:05.820982 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:05.820890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:06.409715 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:06.409664 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:07.050031 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:07.049990 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:28:09.499059 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.499019 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr"] Jul 02 14:28:09.502276 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.502255 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:09.505136 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.505115 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-1c2b7-kube-rbac-proxy-sar-config\"" Jul 02 14:28:09.505256 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.505120 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-1c2b7-serving-cert\"" Jul 02 14:28:09.511717 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.511691 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr"] Jul 02 14:28:09.512598 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.512576 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:09.574136 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.574096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562208bb-2466-484f-a77b-4d413085093a-openshift-service-ca-bundle\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:09.574136 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.574141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:09.675607 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.675563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562208bb-2466-484f-a77b-4d413085093a-openshift-service-ca-bundle\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:09.675781 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.675611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:09.675826 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:28:09.675778 2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-1c2b7-serving-cert: secret "switch-graph-1c2b7-serving-cert" not found Jul 02 14:28:09.675865 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:28:09.675842 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls podName:562208bb-2466-484f-a77b-4d413085093a nodeName:}" failed. No retries permitted until 2026-07-02 14:28:10.175826224 +0000 UTC m=+694.297068442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls") pod "switch-graph-1c2b7-5f799fb869-vb6lr" (UID: "562208bb-2466-484f-a77b-4d413085093a") : secret "switch-graph-1c2b7-serving-cert" not found Jul 02 14:28:09.676323 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.676304 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562208bb-2466-484f-a77b-4d413085093a-openshift-service-ca-bundle\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:09.738649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:09.738617 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:10.179593 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:10.179558 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:10.181992 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:10.181958 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls\") pod \"switch-graph-1c2b7-5f799fb869-vb6lr\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:10.412426 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:10.412388 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:10.544274 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:10.544230 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr"] Jul 02 14:28:10.546708 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:28:10.546681 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562208bb_2466_484f_a77b_4d413085093a.slice/crio-b08d131d275de7119603cd2c52644ace7c233cd4d9be66d257b622153f26e55a WatchSource:0}: Error finding container b08d131d275de7119603cd2c52644ace7c233cd4d9be66d257b622153f26e55a: Status 404 returned error can't find the container with id b08d131d275de7119603cd2c52644ace7c233cd4d9be66d257b622153f26e55a Jul 02 14:28:11.103619 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:11.103572 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" event={"ID":"562208bb-2466-484f-a77b-4d413085093a","Type":"ContainerStarted","Data":"dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061"} Jul 02 14:28:11.103797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:11.103642 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" event={"ID":"562208bb-2466-484f-a77b-4d413085093a","Type":"ContainerStarted","Data":"b08d131d275de7119603cd2c52644ace7c233cd4d9be66d257b622153f26e55a"} Jul 02 14:28:11.103797 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:11.103684 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:11.133522 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:11.133476 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podStartSLOduration=2.133458957 podStartE2EDuration="2.133458957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:28:11.132240007 +0000 UTC m=+695.253482250" watchObservedRunningTime="2026-07-02 14:28:11.133458957 +0000 UTC m=+695.254701198" Jul 02 14:28:12.049778 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:12.049735 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:28:17.050736 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:17.050699 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:28:17.051112 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:17.050800 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:28:17.112476 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:17.112444 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:28:22.050666 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:22.050616 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:28:27.050101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:27.050064 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:28:32.049905 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:32.049825 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:28:35.834345 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:35.834320 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:28:36.003802 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.003716 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d80103d8-904b-4db7-8756-141cc8462374-proxy-tls\") pod \"d80103d8-904b-4db7-8756-141cc8462374\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " Jul 02 14:28:36.003802 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.003757 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d80103d8-904b-4db7-8756-141cc8462374-openshift-service-ca-bundle\") pod \"d80103d8-904b-4db7-8756-141cc8462374\" (UID: \"d80103d8-904b-4db7-8756-141cc8462374\") " Jul 02 14:28:36.004131 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.004109 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d80103d8-904b-4db7-8756-141cc8462374-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d80103d8-904b-4db7-8756-141cc8462374" (UID: "d80103d8-904b-4db7-8756-141cc8462374"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:28:36.006000 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.005976 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d80103d8-904b-4db7-8756-141cc8462374-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d80103d8-904b-4db7-8756-141cc8462374" (UID: "d80103d8-904b-4db7-8756-141cc8462374"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:28:36.104747 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.104719 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d80103d8-904b-4db7-8756-141cc8462374-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:28:36.104747 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.104746 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d80103d8-904b-4db7-8756-141cc8462374-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:28:36.208228 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.208194 2581 generic.go:356] "Generic (PLEG): container finished" podID="d80103d8-904b-4db7-8756-141cc8462374" containerID="1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba" exitCode=0 Jul 02 14:28:36.208412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.208244 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" Jul 02 14:28:36.208412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.208254 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" event={"ID":"d80103d8-904b-4db7-8756-141cc8462374","Type":"ContainerDied","Data":"1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba"} Jul 02 14:28:36.208412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.208295 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s" event={"ID":"d80103d8-904b-4db7-8756-141cc8462374","Type":"ContainerDied","Data":"c40cd1a843418ffda65e93a30a3485f6175da0479f3bbed1f14214fa9e5d7354"} Jul 02 14:28:36.208412 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.208316 2581 scope.go:117] "RemoveContainer" containerID="1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba" Jul 02 14:28:36.216539 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.216521 2581 scope.go:117] "RemoveContainer" containerID="1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba" Jul 02 14:28:36.216790 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:28:36.216774 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba\": container with ID starting with 1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba not found: ID does not exist" containerID="1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba" Jul 02 14:28:36.216839 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.216797 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba"} err="failed to get container status \"1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba\": rpc error: code = NotFound desc = could not find container \"1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba\": container with ID starting with 1785b27159546e6fbb22551eaf109f1383c4259adce63ca049c44dec597314ba not found: ID does not exist" Jul 02 14:28:36.231610 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.231586 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s"] Jul 02 14:28:36.241716 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.237679 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7d6d5bb5c4-kqn2s"] Jul 02 14:28:36.287004 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:36.286925 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d80103d8-904b-4db7-8756-141cc8462374" path="/var/lib/kubelet/pods/d80103d8-904b-4db7-8756-141cc8462374/volumes" Jul 02 14:28:50.674144 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:50.674110 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:28:50.774970 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:28:50.774938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:29:05.390370 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.390339 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r"] Jul 02 14:29:05.390745 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.390623 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" Jul 02 14:29:05.390745 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.390634 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" Jul 02 14:29:05.390745 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.390698 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d80103d8-904b-4db7-8756-141cc8462374" containerName="model-chainer" Jul 02 14:29:05.393335 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.393320 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:05.396988 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.396967 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9a7ca-serving-cert\"" Jul 02 14:29:05.397121 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.396966 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9a7ca-kube-rbac-proxy-sar-config\"" Jul 02 14:29:05.399734 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.399554 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:29:05.408009 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.407985 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r"] Jul 02 14:29:05.423676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.423643 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:05.423806 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.423687 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa929eaf-731e-484c-902a-ecf4cbdbd504-openshift-service-ca-bundle\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:05.524360 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.524327 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:05.524544 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.524367 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa929eaf-731e-484c-902a-ecf4cbdbd504-openshift-service-ca-bundle\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:05.524544 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:29:05.524464 2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-9a7ca-serving-cert: secret "sequence-graph-9a7ca-serving-cert" not found Jul 02 14:29:05.524544 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:29:05.524529 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls podName:fa929eaf-731e-484c-902a-ecf4cbdbd504 nodeName:}" failed. No retries permitted until 2026-07-02 14:29:06.024513231 +0000 UTC m=+750.145755448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls") pod "sequence-graph-9a7ca-7695d79cf4-h9l2r" (UID: "fa929eaf-731e-484c-902a-ecf4cbdbd504") : secret "sequence-graph-9a7ca-serving-cert" not found Jul 02 14:29:05.524950 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.524933 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa929eaf-731e-484c-902a-ecf4cbdbd504-openshift-service-ca-bundle\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:05.617957 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:05.617924 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:29:06.028857 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:06.028821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:06.031194 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:06.031173 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls\") pod \"sequence-graph-9a7ca-7695d79cf4-h9l2r\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:06.303111 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:06.303041 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:06.433066 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:06.432889 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r"] Jul 02 14:29:06.436131 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:29:06.436104 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa929eaf_731e_484c_902a_ecf4cbdbd504.slice/crio-1952053bb605bbb485bf0a36e4e746ab55928e4609182bdc197da295aa0c7447 WatchSource:0}: Error finding container 1952053bb605bbb485bf0a36e4e746ab55928e4609182bdc197da295aa0c7447: Status 404 returned error can't find the container with id 1952053bb605bbb485bf0a36e4e746ab55928e4609182bdc197da295aa0c7447 Jul 02 14:29:07.335479 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:07.335442 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" event={"ID":"fa929eaf-731e-484c-902a-ecf4cbdbd504","Type":"ContainerStarted","Data":"5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec"} Jul 02 14:29:07.335647 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:07.335496 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" event={"ID":"fa929eaf-731e-484c-902a-ecf4cbdbd504","Type":"ContainerStarted","Data":"1952053bb605bbb485bf0a36e4e746ab55928e4609182bdc197da295aa0c7447"} Jul 02 14:29:07.335647 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:07.335523 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:07.354107 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:07.354053 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podStartSLOduration=2.354038425 podStartE2EDuration="2.354038425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:29:07.352531966 +0000 UTC m=+751.473774206" watchObservedRunningTime="2026-07-02 14:29:07.354038425 +0000 UTC m=+751.475280665" Jul 02 14:29:13.345425 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:13.345399 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:29:58.553813 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:29:58.553741 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:31:35.851723 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:31:35.851644 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:31:36.279839 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:31:36.279803 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:31:36.283009 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:31:36.282987 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:31:44.423978 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:31:44.423943 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:31:44.505604 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:31:44.505567 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:33:08.944393 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:33:08.944313 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:35:18.908002 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:35:18.907968 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:24.254272 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.254237 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr"] Jul 02 14:36:24.254810 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.254430 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" containerID="cri-o://dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061" gracePeriod=30 Jul 02 14:36:24.293191 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.293162 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:24.491770 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.491742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:24.564812 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.564733 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:24.643382 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.643349 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:24.745040 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.745009 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:24.889663 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:24.889636 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:36:27.111237 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:27.111195 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:36:32.111406 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:32.111363 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:36:36.304752 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:36.304724 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:36:36.307346 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:36.307323 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:36:37.111330 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:37.111289 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:36:37.111498 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:37.111393 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:36:42.110834 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:42.110795 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:36:47.111419 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:47.111380 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:36:52.111304 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:52.111261 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:36:54.400303 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.400275 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:36:54.507120 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.507079 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562208bb-2466-484f-a77b-4d413085093a-openshift-service-ca-bundle\") pod \"562208bb-2466-484f-a77b-4d413085093a\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " Jul 02 14:36:54.507337 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.507135 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls\") pod \"562208bb-2466-484f-a77b-4d413085093a\" (UID: \"562208bb-2466-484f-a77b-4d413085093a\") " Jul 02 14:36:54.507455 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.507433 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/562208bb-2466-484f-a77b-4d413085093a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "562208bb-2466-484f-a77b-4d413085093a" (UID: "562208bb-2466-484f-a77b-4d413085093a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:36:54.509131 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.509087 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "562208bb-2466-484f-a77b-4d413085093a" (UID: "562208bb-2466-484f-a77b-4d413085093a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:36:54.608724 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.608684 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/562208bb-2466-484f-a77b-4d413085093a-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:36:54.608724 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:54.608716 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/562208bb-2466-484f-a77b-4d413085093a-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:36:55.262417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.262383 2581 generic.go:356] "Generic (PLEG): container finished" podID="562208bb-2466-484f-a77b-4d413085093a" containerID="dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061" exitCode=0 Jul 02 14:36:55.262708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.262435 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" Jul 02 14:36:55.262708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.262467 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" event={"ID":"562208bb-2466-484f-a77b-4d413085093a","Type":"ContainerDied","Data":"dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061"} Jul 02 14:36:55.262708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.262521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr" event={"ID":"562208bb-2466-484f-a77b-4d413085093a","Type":"ContainerDied","Data":"b08d131d275de7119603cd2c52644ace7c233cd4d9be66d257b622153f26e55a"} Jul 02 14:36:55.262708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.262544 2581 scope.go:117] "RemoveContainer" containerID="dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061" Jul 02 14:36:55.270678 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.270662 2581 scope.go:117] "RemoveContainer" containerID="dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061" Jul 02 14:36:55.270926 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:36:55.270910 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061\": container with ID starting with dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061 not found: ID does not exist" containerID="dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061" Jul 02 14:36:55.270974 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.270933 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061"} err="failed to get container status \"dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061\": rpc error: code = NotFound desc = could not find container \"dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061\": container with ID starting with dc4b173391258dfec47e806234255180ef4aa9c1598b206dfbc49ae2597e6061 not found: ID does not exist" Jul 02 14:36:55.284582 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.284548 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr"] Jul 02 14:36:55.289032 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:55.289012 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1c2b7-5f799fb869-vb6lr"] Jul 02 14:36:56.285492 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:36:56.285457 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562208bb-2466-484f-a77b-4d413085093a" path="/var/lib/kubelet/pods/562208bb-2466-484f-a77b-4d413085093a/volumes" Jul 02 14:37:20.083822 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.083783 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r"] Jul 02 14:37:20.084263 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.083991 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" containerID="cri-o://5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec" gracePeriod=30 Jul 02 14:37:20.100195 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.100144 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:20.214053 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.214016 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:20.314910 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.314875 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:20.417049 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.417017 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:20.528684 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.528651 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:20.595008 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:20.594974 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:23.343506 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:23.343461 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:24.470862 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.470831 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv"] Jul 02 14:37:24.471266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.471146 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" Jul 02 14:37:24.471266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.471171 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" Jul 02 14:37:24.471266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.471222 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="562208bb-2466-484f-a77b-4d413085093a" containerName="switch-graph-1c2b7" Jul 02 14:37:24.473975 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.473959 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.476477 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.476450 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-fbc8b-kube-rbac-proxy-sar-config\"" Jul 02 14:37:24.476603 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.476456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-fbc8b-serving-cert\"" Jul 02 14:37:24.485518 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.485492 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv"] Jul 02 14:37:24.486312 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.486287 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:24.563305 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.563270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db040183-bf2c-450e-9c88-7d6a63a87995-proxy-tls\") pod \"ensemble-graph-fbc8b-79dc459f7b-9p9rv\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.563488 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.563327 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db040183-bf2c-450e-9c88-7d6a63a87995-openshift-service-ca-bundle\") pod \"ensemble-graph-fbc8b-79dc459f7b-9p9rv\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.664445 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.664410 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db040183-bf2c-450e-9c88-7d6a63a87995-proxy-tls\") pod \"ensemble-graph-fbc8b-79dc459f7b-9p9rv\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.664625 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.664464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db040183-bf2c-450e-9c88-7d6a63a87995-openshift-service-ca-bundle\") pod \"ensemble-graph-fbc8b-79dc459f7b-9p9rv\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.665061 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.665040 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db040183-bf2c-450e-9c88-7d6a63a87995-openshift-service-ca-bundle\") pod \"ensemble-graph-fbc8b-79dc459f7b-9p9rv\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.666851 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.666832 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db040183-bf2c-450e-9c88-7d6a63a87995-proxy-tls\") pod \"ensemble-graph-fbc8b-79dc459f7b-9p9rv\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.695684 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.695648 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:24.783844 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.783766 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:24.914430 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:24.914405 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv"] Jul 02 14:37:24.916123 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:37:24.916095 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb040183_bf2c_450e_9c88_7d6a63a87995.slice/crio-bba85900ce74aec8e28350a60b877ac2341091a6eb71cf1a32dd7342981d5e8f WatchSource:0}: Error finding container bba85900ce74aec8e28350a60b877ac2341091a6eb71cf1a32dd7342981d5e8f: Status 404 returned error can't find the container with id bba85900ce74aec8e28350a60b877ac2341091a6eb71cf1a32dd7342981d5e8f Jul 02 14:37:25.378733 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:25.378693 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" event={"ID":"db040183-bf2c-450e-9c88-7d6a63a87995","Type":"ContainerStarted","Data":"a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5"} Jul 02 14:37:25.378922 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:25.378752 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" event={"ID":"db040183-bf2c-450e-9c88-7d6a63a87995","Type":"ContainerStarted","Data":"bba85900ce74aec8e28350a60b877ac2341091a6eb71cf1a32dd7342981d5e8f"} Jul 02 14:37:25.378922 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:25.378829 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:25.395503 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:25.395444 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podStartSLOduration=1.395425774 podStartE2EDuration="1.395425774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:37:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:37:25.395076049 +0000 UTC m=+1249.516318314" watchObservedRunningTime="2026-07-02 14:37:25.395425774 +0000 UTC m=+1249.516668015" Jul 02 14:37:28.343482 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:28.343386 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:31.389102 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:31.389060 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:33.343729 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:33.343692 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:33.344117 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:33.343798 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:37:34.556708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:34.556633 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv"] Jul 02 14:37:34.557076 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:34.556892 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" containerID="cri-o://a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5" gracePeriod=30 Jul 02 14:37:34.577464 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:34.577438 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:34.802888 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:34.802853 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:34.868726 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:34.868651 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:34.945478 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:34.945442 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:35.110509 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:35.110481 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:37:36.386686 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:36.386653 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:38.343120 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:38.343077 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:41.386452 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:41.386412 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:43.343438 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:43.343391 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:46.386403 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:46.386364 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:46.386861 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:46.386471 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:37:48.343133 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:48.343089 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:50.261678 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.261655 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:37:50.362510 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.362430 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls\") pod \"fa929eaf-731e-484c-902a-ecf4cbdbd504\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " Jul 02 14:37:50.362510 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.362470 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa929eaf-731e-484c-902a-ecf4cbdbd504-openshift-service-ca-bundle\") pod \"fa929eaf-731e-484c-902a-ecf4cbdbd504\" (UID: \"fa929eaf-731e-484c-902a-ecf4cbdbd504\") " Jul 02 14:37:50.362839 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.362816 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa929eaf-731e-484c-902a-ecf4cbdbd504-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "fa929eaf-731e-484c-902a-ecf4cbdbd504" (UID: "fa929eaf-731e-484c-902a-ecf4cbdbd504"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:37:50.364460 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.364438 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "fa929eaf-731e-484c-902a-ecf4cbdbd504" (UID: "fa929eaf-731e-484c-902a-ecf4cbdbd504"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:37:50.463524 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.463490 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fa929eaf-731e-484c-902a-ecf4cbdbd504-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:37:50.463524 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.463519 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa929eaf-731e-484c-902a-ecf4cbdbd504-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:37:50.477008 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.476978 2581 generic.go:356] "Generic (PLEG): container finished" podID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerID="5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec" exitCode=0 Jul 02 14:37:50.477138 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.477025 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" Jul 02 14:37:50.477138 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.477061 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" event={"ID":"fa929eaf-731e-484c-902a-ecf4cbdbd504","Type":"ContainerDied","Data":"5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec"} Jul 02 14:37:50.477138 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.477110 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r" event={"ID":"fa929eaf-731e-484c-902a-ecf4cbdbd504","Type":"ContainerDied","Data":"1952053bb605bbb485bf0a36e4e746ab55928e4609182bdc197da295aa0c7447"} Jul 02 14:37:50.477138 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.477134 2581 scope.go:117] "RemoveContainer" containerID="5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec" Jul 02 14:37:50.485539 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.485522 2581 scope.go:117] "RemoveContainer" containerID="5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec" Jul 02 14:37:50.485818 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:37:50.485799 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec\": container with ID starting with 5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec not found: ID does not exist" containerID="5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec" Jul 02 14:37:50.485866 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.485827 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec"} err="failed to get container status \"5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec\": rpc error: code = NotFound desc = could not find container \"5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec\": container with ID starting with 5f4567a3f085d9e3e6755a038958ca799488722d7a8e27b09b97d2d81e8951ec not found: ID does not exist" Jul 02 14:37:50.498966 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.498942 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r"] Jul 02 14:37:50.504037 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:50.504015 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9a7ca-7695d79cf4-h9l2r"] Jul 02 14:37:51.387167 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:51.387127 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:37:52.284865 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:52.284832 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" path="/var/lib/kubelet/pods/fa929eaf-731e-484c-902a-ecf4cbdbd504/volumes" Jul 02 14:37:56.386333 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:37:56.386298 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:38:01.386814 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:01.386771 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:38:04.701236 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.701208 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:38:04.777820 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.777780 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db040183-bf2c-450e-9c88-7d6a63a87995-openshift-service-ca-bundle\") pod \"db040183-bf2c-450e-9c88-7d6a63a87995\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " Jul 02 14:38:04.778008 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.777833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db040183-bf2c-450e-9c88-7d6a63a87995-proxy-tls\") pod \"db040183-bf2c-450e-9c88-7d6a63a87995\" (UID: \"db040183-bf2c-450e-9c88-7d6a63a87995\") " Jul 02 14:38:04.778181 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.778136 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db040183-bf2c-450e-9c88-7d6a63a87995-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "db040183-bf2c-450e-9c88-7d6a63a87995" (UID: "db040183-bf2c-450e-9c88-7d6a63a87995"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:38:04.779781 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.779752 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db040183-bf2c-450e-9c88-7d6a63a87995-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "db040183-bf2c-450e-9c88-7d6a63a87995" (UID: "db040183-bf2c-450e-9c88-7d6a63a87995"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:38:04.879260 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.879228 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db040183-bf2c-450e-9c88-7d6a63a87995-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:38:04.879260 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:04.879260 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/db040183-bf2c-450e-9c88-7d6a63a87995-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:38:05.538920 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.538885 2581 generic.go:356] "Generic (PLEG): container finished" podID="db040183-bf2c-450e-9c88-7d6a63a87995" containerID="a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5" exitCode=0 Jul 02 14:38:05.539112 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.538933 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" Jul 02 14:38:05.539112 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.538946 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" event={"ID":"db040183-bf2c-450e-9c88-7d6a63a87995","Type":"ContainerDied","Data":"a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5"} Jul 02 14:38:05.539112 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.539013 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv" event={"ID":"db040183-bf2c-450e-9c88-7d6a63a87995","Type":"ContainerDied","Data":"bba85900ce74aec8e28350a60b877ac2341091a6eb71cf1a32dd7342981d5e8f"} Jul 02 14:38:05.539112 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.539045 2581 scope.go:117] "RemoveContainer" containerID="a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5" Jul 02 14:38:05.547320 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.547305 2581 scope.go:117] "RemoveContainer" containerID="a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5" Jul 02 14:38:05.547543 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:38:05.547524 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5\": container with ID starting with a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5 not found: ID does not exist" containerID="a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5" Jul 02 14:38:05.547589 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.547551 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5"} err="failed to get container status \"a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5\": rpc error: code = NotFound desc = could not find container \"a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5\": container with ID starting with a5965876d2e47db2b6ddce9a6e7c8055114086a233f8676aa4e4c67e051f3fa5 not found: ID does not exist" Jul 02 14:38:05.564329 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.564302 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv"] Jul 02 14:38:05.568476 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:05.568454 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fbc8b-79dc459f7b-9p9rv"] Jul 02 14:38:06.285121 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:06.285089 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" path="/var/lib/kubelet/pods/db040183-bf2c-450e-9c88-7d6a63a87995/volumes" Jul 02 14:38:30.274880 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.274852 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq"] Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.275217 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.275235 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.275248 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.275257 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.275359 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db040183-bf2c-450e-9c88-7d6a63a87995" containerName="ensemble-graph-fbc8b" Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.275378 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa929eaf-731e-484c-902a-ecf4cbdbd504" containerName="sequence-graph-9a7ca" Jul 02 14:38:30.277417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.276694 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:30.278219 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.278200 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq"] Jul 02 14:38:30.278320 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.278277 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.280821 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.280801 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8937c-serving-cert\"" Jul 02 14:38:30.281060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.281045 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fn24b\"" Jul 02 14:38:30.281137 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.281063 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 02 14:38:30.281225 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.281201 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-8937c-kube-rbac-proxy-sar-config\"" Jul 02 14:38:30.389998 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.389962 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-openshift-service-ca-bundle\") pod \"sequence-graph-8937c-7586b86695-n6mlq\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.390192 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.390009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-proxy-tls\") pod \"sequence-graph-8937c-7586b86695-n6mlq\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.491335 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.491284 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-openshift-service-ca-bundle\") pod \"sequence-graph-8937c-7586b86695-n6mlq\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.491335 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.491347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-proxy-tls\") pod \"sequence-graph-8937c-7586b86695-n6mlq\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.491931 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.491909 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-openshift-service-ca-bundle\") pod \"sequence-graph-8937c-7586b86695-n6mlq\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.493653 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.493635 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-proxy-tls\") pod \"sequence-graph-8937c-7586b86695-n6mlq\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.588826 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.588753 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:30.714895 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:30.714869 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq"] Jul 02 14:38:30.717275 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:38:30.717249 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb34f367_3b5f_4a8f_a70e_1db545a1ca9a.slice/crio-bb3c086c9dbada68dfd3504e1e559793435c3706755c8c100a68bde681180d8e WatchSource:0}: Error finding container bb3c086c9dbada68dfd3504e1e559793435c3706755c8c100a68bde681180d8e: Status 404 returned error can't find the container with id bb3c086c9dbada68dfd3504e1e559793435c3706755c8c100a68bde681180d8e Jul 02 14:38:31.443377 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:31.443338 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:31.644914 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:31.644881 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" event={"ID":"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a","Type":"ContainerStarted","Data":"78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7"} Jul 02 14:38:31.645095 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:31.644942 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" event={"ID":"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a","Type":"ContainerStarted","Data":"bb3c086c9dbada68dfd3504e1e559793435c3706755c8c100a68bde681180d8e"} Jul 02 14:38:31.645095 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:31.645058 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:31.662271 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:31.662219 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podStartSLOduration=1.662203521 podStartE2EDuration="1.662203521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:38:31.660485951 +0000 UTC m=+1315.781728190" watchObservedRunningTime="2026-07-02 14:38:31.662203521 +0000 UTC m=+1315.783445760" Jul 02 14:38:34.772287 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.772255 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch"] Jul 02 14:38:34.775400 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.775383 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:34.777702 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.777684 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-57d01-serving-cert\"" Jul 02 14:38:34.777922 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.777907 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-57d01-kube-rbac-proxy-sar-config\"" Jul 02 14:38:34.785375 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.785354 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch"] Jul 02 14:38:34.786261 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.786239 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:34.830083 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.830054 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-openshift-service-ca-bundle\") pod \"ensemble-graph-57d01-54cddf9746-zqjch\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:34.830243 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.830099 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-proxy-tls\") pod \"ensemble-graph-57d01-54cddf9746-zqjch\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:34.895234 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.895201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:34.931294 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.931268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-openshift-service-ca-bundle\") pod \"ensemble-graph-57d01-54cddf9746-zqjch\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:34.931440 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.931308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-proxy-tls\") pod \"ensemble-graph-57d01-54cddf9746-zqjch\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:34.931946 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.931922 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-openshift-service-ca-bundle\") pod \"ensemble-graph-57d01-54cddf9746-zqjch\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:34.933551 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:34.933532 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-proxy-tls\") pod \"ensemble-graph-57d01-54cddf9746-zqjch\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:35.085691 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:35.085603 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:35.220793 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:35.220763 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch"] Jul 02 14:38:35.223432 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:38:35.223402 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1fee85a_ce18_47e7_8f0a_74efdf5004d7.slice/crio-0fa5805226fa34737b546e4f4b4504cf6cae09b70d8afd64c6d72122aa64d973 WatchSource:0}: Error finding container 0fa5805226fa34737b546e4f4b4504cf6cae09b70d8afd64c6d72122aa64d973: Status 404 returned error can't find the container with id 0fa5805226fa34737b546e4f4b4504cf6cae09b70d8afd64c6d72122aa64d973 Jul 02 14:38:35.662129 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:35.662086 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" event={"ID":"c1fee85a-ce18-47e7-8f0a-74efdf5004d7","Type":"ContainerStarted","Data":"46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4"} Jul 02 14:38:35.662355 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:35.662179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" event={"ID":"c1fee85a-ce18-47e7-8f0a-74efdf5004d7","Type":"ContainerStarted","Data":"0fa5805226fa34737b546e4f4b4504cf6cae09b70d8afd64c6d72122aa64d973"} Jul 02 14:38:35.662355 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:35.662210 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:35.678827 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:35.678779 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podStartSLOduration=1.6787641899999999 podStartE2EDuration="1.67876419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:38:35.678223982 +0000 UTC m=+1319.799466224" watchObservedRunningTime="2026-07-02 14:38:35.67876419 +0000 UTC m=+1319.800006430" Jul 02 14:38:37.655309 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:37.655281 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:38:41.672388 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:41.672355 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:38:44.438520 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:44.438480 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq"] Jul 02 14:38:44.438975 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:44.438697 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" containerID="cri-o://78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7" gracePeriod=30 Jul 02 14:38:44.479640 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:44.479599 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:44.615316 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:44.615271 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:44.709871 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:44.709792 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:44.809337 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:44.809304 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:45.126229 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:45.126201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:45.257060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:45.257014 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:38:47.653194 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:47.653141 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:38:52.653527 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:52.653481 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:38:57.653899 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:57.653864 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:38:57.654366 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:38:57.653989 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:39:02.653552 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:02.653513 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:39:07.653360 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:07.653321 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:39:12.653730 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:12.653692 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:39:14.588572 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.588545 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:39:14.621269 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.621228 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-openshift-service-ca-bundle\") pod \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " Jul 02 14:39:14.621450 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.621278 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-proxy-tls\") pod \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\" (UID: \"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a\") " Jul 02 14:39:14.621615 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.621588 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" (UID: "cb34f367-3b5f-4a8f-a70e-1db545a1ca9a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:39:14.623466 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.623438 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" (UID: "cb34f367-3b5f-4a8f-a70e-1db545a1ca9a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:39:14.722533 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.722452 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:39:14.722533 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.722479 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:39:14.815168 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.815129 2581 generic.go:356] "Generic (PLEG): container finished" podID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerID="78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7" exitCode=0 Jul 02 14:39:14.815339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.815192 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" Jul 02 14:39:14.815339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.815192 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" event={"ID":"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a","Type":"ContainerDied","Data":"78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7"} Jul 02 14:39:14.815339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.815302 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq" event={"ID":"cb34f367-3b5f-4a8f-a70e-1db545a1ca9a","Type":"ContainerDied","Data":"bb3c086c9dbada68dfd3504e1e559793435c3706755c8c100a68bde681180d8e"} Jul 02 14:39:14.815339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.815329 2581 scope.go:117] "RemoveContainer" containerID="78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7" Jul 02 14:39:14.823757 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.823741 2581 scope.go:117] "RemoveContainer" containerID="78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7" Jul 02 14:39:14.823986 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:39:14.823969 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7\": container with ID starting with 78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7 not found: ID does not exist" containerID="78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7" Jul 02 14:39:14.824027 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.823995 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7"} err="failed to get container status \"78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7\": rpc error: code = NotFound desc = could not find container \"78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7\": container with ID starting with 78056232d5a3c6c3b65d4de64c800e4c85b887085192de87ffcbb82471e23bb7 not found: ID does not exist" Jul 02 14:39:14.836987 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.836963 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq"] Jul 02 14:39:14.842723 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:14.842705 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-8937c-7586b86695-n6mlq"] Jul 02 14:39:16.285042 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:16.285008 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" path="/var/lib/kubelet/pods/cb34f367-3b5f-4a8f-a70e-1db545a1ca9a/volumes" Jul 02 14:39:44.670617 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.670584 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99"] Jul 02 14:39:44.671048 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.670894 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" Jul 02 14:39:44.671048 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.670906 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" Jul 02 14:39:44.671048 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.670975 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb34f367-3b5f-4a8f-a70e-1db545a1ca9a" containerName="sequence-graph-8937c" Jul 02 14:39:44.675120 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.675104 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.677661 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.677639 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c3f32-kube-rbac-proxy-sar-config\"" Jul 02 14:39:44.677786 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.677642 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c3f32-serving-cert\"" Jul 02 14:39:44.679132 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.679110 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:39:44.682982 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.682962 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99"] Jul 02 14:39:44.766183 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.766130 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/828bdff3-9b07-4549-bf67-d8c99187a2f9-proxy-tls\") pod \"sequence-graph-c3f32-d7476f4ff-9nl99\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.766339 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.766209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/828bdff3-9b07-4549-bf67-d8c99187a2f9-openshift-service-ca-bundle\") pod \"sequence-graph-c3f32-d7476f4ff-9nl99\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.840545 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.840465 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:39:44.866646 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.866623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/828bdff3-9b07-4549-bf67-d8c99187a2f9-proxy-tls\") pod \"sequence-graph-c3f32-d7476f4ff-9nl99\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.866784 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.866665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/828bdff3-9b07-4549-bf67-d8c99187a2f9-openshift-service-ca-bundle\") pod \"sequence-graph-c3f32-d7476f4ff-9nl99\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.867263 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.867246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/828bdff3-9b07-4549-bf67-d8c99187a2f9-openshift-service-ca-bundle\") pod \"sequence-graph-c3f32-d7476f4ff-9nl99\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.869005 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.868985 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/828bdff3-9b07-4549-bf67-d8c99187a2f9-proxy-tls\") pod \"sequence-graph-c3f32-d7476f4ff-9nl99\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:44.986000 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:44.985929 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:45.111876 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:45.111850 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99"] Jul 02 14:39:45.114498 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:39:45.114469 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828bdff3_9b07_4549_bf67_d8c99187a2f9.slice/crio-51ac3a9647c3c74361d383c5fa2d21ea3fda12eb820f194ad3468007b1c09439 WatchSource:0}: Error finding container 51ac3a9647c3c74361d383c5fa2d21ea3fda12eb820f194ad3468007b1c09439: Status 404 returned error can't find the container with id 51ac3a9647c3c74361d383c5fa2d21ea3fda12eb820f194ad3468007b1c09439 Jul 02 14:39:45.951238 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:45.951204 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" event={"ID":"828bdff3-9b07-4549-bf67-d8c99187a2f9","Type":"ContainerStarted","Data":"7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed"} Jul 02 14:39:45.951734 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:45.951261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" event={"ID":"828bdff3-9b07-4549-bf67-d8c99187a2f9","Type":"ContainerStarted","Data":"51ac3a9647c3c74361d383c5fa2d21ea3fda12eb820f194ad3468007b1c09439"} Jul 02 14:39:45.951734 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:45.951327 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:39:45.968619 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:45.968570 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podStartSLOduration=1.968555293 podStartE2EDuration="1.968555293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:39:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:39:45.966691275 +0000 UTC m=+1390.087933516" watchObservedRunningTime="2026-07-02 14:39:45.968555293 +0000 UTC m=+1390.089797560" Jul 02 14:39:51.961994 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:39:51.961964 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:40:36.234746 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:40:36.234714 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:41:36.333835 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:41:36.333806 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:41:36.339433 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:41:36.339412 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:42:57.462958 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:42:57.462929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:43:06.006084 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:43:06.006057 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:05.491968 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:05.491926 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:36.363351 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:36.363311 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:46:36.367028 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:36.367003 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:46:49.471417 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.471384 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch"] Jul 02 14:46:49.471802 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.471592 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" containerID="cri-o://46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4" gracePeriod=30 Jul 02 14:46:49.503099 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.503066 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:49.620254 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.620202 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:49.704105 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.704073 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:49.781394 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.781311 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:49.958754 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:49.958723 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:50.053691 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:50.053603 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:50.379914 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:50.379881 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:46:51.670902 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:51.670864 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:46:56.670165 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:46:56.670112 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:47:01.670589 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:01.670546 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:47:01.671101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:01.670673 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:47:06.670597 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:06.670553 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:47:11.670538 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:11.670500 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:47:16.670927 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:16.670889 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:47:19.606475 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.606434 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:47:19.774338 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.774250 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-openshift-service-ca-bundle\") pod \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " Jul 02 14:47:19.774338 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.774293 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-proxy-tls\") pod \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\" (UID: \"c1fee85a-ce18-47e7-8f0a-74efdf5004d7\") " Jul 02 14:47:19.774654 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.774632 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c1fee85a-ce18-47e7-8f0a-74efdf5004d7" (UID: "c1fee85a-ce18-47e7-8f0a-74efdf5004d7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:47:19.776377 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.776356 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c1fee85a-ce18-47e7-8f0a-74efdf5004d7" (UID: "c1fee85a-ce18-47e7-8f0a-74efdf5004d7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:47:19.820923 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.820890 2581 generic.go:356] "Generic (PLEG): container finished" podID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerID="46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4" exitCode=0 Jul 02 14:47:19.821060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.820932 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" Jul 02 14:47:19.821060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.820964 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" event={"ID":"c1fee85a-ce18-47e7-8f0a-74efdf5004d7","Type":"ContainerDied","Data":"46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4"} Jul 02 14:47:19.821060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.821016 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch" event={"ID":"c1fee85a-ce18-47e7-8f0a-74efdf5004d7","Type":"ContainerDied","Data":"0fa5805226fa34737b546e4f4b4504cf6cae09b70d8afd64c6d72122aa64d973"} Jul 02 14:47:19.821060 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.821037 2581 scope.go:117] "RemoveContainer" containerID="46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4" Jul 02 14:47:19.831296 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.831279 2581 scope.go:117] "RemoveContainer" containerID="46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4" Jul 02 14:47:19.831562 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:47:19.831543 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4\": container with ID starting with 46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4 not found: ID does not exist" containerID="46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4" Jul 02 14:47:19.831630 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.831574 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4"} err="failed to get container status \"46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4\": rpc error: code = NotFound desc = could not find container \"46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4\": container with ID starting with 46358546a4fe426391af24438c5c564593bb7e9b802b6559783a4e3e885abca4 not found: ID does not exist" Jul 02 14:47:19.845920 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.845889 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch"] Jul 02 14:47:19.849851 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.849831 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-57d01-54cddf9746-zqjch"] Jul 02 14:47:19.875853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.875822 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:47:19.875853 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:19.875849 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c1fee85a-ce18-47e7-8f0a-74efdf5004d7-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:47:20.285293 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:20.285265 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" path="/var/lib/kubelet/pods/c1fee85a-ce18-47e7-8f0a-74efdf5004d7/volumes" Jul 02 14:47:49.709360 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.709318 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m"] Jul 02 14:47:49.709838 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.709738 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" Jul 02 14:47:49.709838 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.709756 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" Jul 02 14:47:49.709950 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.709870 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1fee85a-ce18-47e7-8f0a-74efdf5004d7" containerName="ensemble-graph-57d01" Jul 02 14:47:49.712754 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.712734 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:49.715258 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.715233 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-484ca-kube-rbac-proxy-sar-config\"" Jul 02 14:47:49.715474 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.715459 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-484ca-serving-cert\"" Jul 02 14:47:49.716202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.716175 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:49.725492 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.725472 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m"] Jul 02 14:47:49.810411 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.810379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:49.810592 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.810457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-openshift-service-ca-bundle\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:49.911873 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.911836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-openshift-service-ca-bundle\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:49.912023 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.911914 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:49.912066 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:47:49.912046 2581 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-484ca-serving-cert: secret "splitter-graph-484ca-serving-cert" not found Jul 02 14:47:49.912142 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:47:49.912129 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls podName:54a22088-ebd6-43b6-bd9b-897d5f5eda7c nodeName:}" failed. No retries permitted until 2026-07-02 14:47:50.412103806 +0000 UTC m=+1874.533346025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls") pod "splitter-graph-484ca-db8ddbcc6-fjb5m" (UID: "54a22088-ebd6-43b6-bd9b-897d5f5eda7c") : secret "splitter-graph-484ca-serving-cert" not found Jul 02 14:47:49.912616 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:49.912596 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-openshift-service-ca-bundle\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:50.107086 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.107012 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:50.415981 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.415945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:50.418251 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.418231 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls\") pod \"splitter-graph-484ca-db8ddbcc6-fjb5m\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:50.622963 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.622927 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:50.748327 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.748305 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m"] Jul 02 14:47:50.751089 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:47:50.751063 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a22088_ebd6_43b6_bd9b_897d5f5eda7c.slice/crio-75e37d77cde615203c9e0fe9986cd7e82527ff05835cb0fa941fb0df74a64f72 WatchSource:0}: Error finding container 75e37d77cde615203c9e0fe9986cd7e82527ff05835cb0fa941fb0df74a64f72: Status 404 returned error can't find the container with id 75e37d77cde615203c9e0fe9986cd7e82527ff05835cb0fa941fb0df74a64f72 Jul 02 14:47:50.947465 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.947367 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" event={"ID":"54a22088-ebd6-43b6-bd9b-897d5f5eda7c","Type":"ContainerStarted","Data":"74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256"} Jul 02 14:47:50.947465 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.947428 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" event={"ID":"54a22088-ebd6-43b6-bd9b-897d5f5eda7c","Type":"ContainerStarted","Data":"75e37d77cde615203c9e0fe9986cd7e82527ff05835cb0fa941fb0df74a64f72"} Jul 02 14:47:50.947659 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.947472 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:50.965012 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:50.964964 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podStartSLOduration=1.964949643 podStartE2EDuration="1.964949643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:47:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:47:50.96349535 +0000 UTC m=+1875.084737592" watchObservedRunningTime="2026-07-02 14:47:50.964949643 +0000 UTC m=+1875.086191884" Jul 02 14:47:56.957052 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:56.957023 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:47:59.253315 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.253228 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99"] Jul 02 14:47:59.253751 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.253484 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" containerID="cri-o://7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed" gracePeriod=30 Jul 02 14:47:59.295407 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.295371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.398477 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.398449 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.498921 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.498883 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.596200 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.596097 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.696262 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.696231 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.778500 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.778472 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m"] Jul 02 14:47:59.778677 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.778657 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" containerID="cri-o://74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256" gracePeriod=30 Jul 02 14:47:59.801256 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.801225 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.886815 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.886774 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:47:59.970050 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:47:59.970002 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:48:00.047881 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:00.047850 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:48:00.164428 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:00.164323 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:48:00.260754 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:00.260709 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:48:01.955793 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:01.955748 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:01.958842 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:01.958813 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:06.955622 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:06.955582 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:06.958485 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:06.958459 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:11.955719 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:11.955679 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:11.956089 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:11.955785 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:48:11.958723 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:11.958697 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:11.958826 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:11.958775 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:48:16.955927 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:16.955885 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:16.958790 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:16.958760 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:21.955222 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:21.955180 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:21.958356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:21.958326 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:26.955420 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:26.955372 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:26.958510 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:26.958472 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:48:29.394835 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.394808 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:48:29.442708 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.442680 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/828bdff3-9b07-4549-bf67-d8c99187a2f9-proxy-tls\") pod \"828bdff3-9b07-4549-bf67-d8c99187a2f9\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " Jul 02 14:48:29.444682 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.444650 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/828bdff3-9b07-4549-bf67-d8c99187a2f9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "828bdff3-9b07-4549-bf67-d8c99187a2f9" (UID: "828bdff3-9b07-4549-bf67-d8c99187a2f9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:48:29.543185 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.543063 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/828bdff3-9b07-4549-bf67-d8c99187a2f9-openshift-service-ca-bundle\") pod \"828bdff3-9b07-4549-bf67-d8c99187a2f9\" (UID: \"828bdff3-9b07-4549-bf67-d8c99187a2f9\") " Jul 02 14:48:29.543350 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.543255 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/828bdff3-9b07-4549-bf67-d8c99187a2f9-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:48:29.543456 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.543431 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/828bdff3-9b07-4549-bf67-d8c99187a2f9-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "828bdff3-9b07-4549-bf67-d8c99187a2f9" (UID: "828bdff3-9b07-4549-bf67-d8c99187a2f9"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:48:29.644042 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.644007 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/828bdff3-9b07-4549-bf67-d8c99187a2f9-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:48:29.919183 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.919143 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:48:29.946585 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.946560 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls\") pod \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " Jul 02 14:48:29.946749 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.946590 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-openshift-service-ca-bundle\") pod \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\" (UID: \"54a22088-ebd6-43b6-bd9b-897d5f5eda7c\") " Jul 02 14:48:29.946982 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.946960 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "54a22088-ebd6-43b6-bd9b-897d5f5eda7c" (UID: "54a22088-ebd6-43b6-bd9b-897d5f5eda7c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:48:29.948514 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:29.948494 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "54a22088-ebd6-43b6-bd9b-897d5f5eda7c" (UID: "54a22088-ebd6-43b6-bd9b-897d5f5eda7c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:48:30.047272 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.047148 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:48:30.047272 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.047220 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54a22088-ebd6-43b6-bd9b-897d5f5eda7c-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:48:30.118205 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.118165 2581 generic.go:356] "Generic (PLEG): container finished" podID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerID="7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed" exitCode=0 Jul 02 14:48:30.118388 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.118227 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" Jul 02 14:48:30.118388 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.118249 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" event={"ID":"828bdff3-9b07-4549-bf67-d8c99187a2f9","Type":"ContainerDied","Data":"7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed"} Jul 02 14:48:30.118388 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.118321 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99" event={"ID":"828bdff3-9b07-4549-bf67-d8c99187a2f9","Type":"ContainerDied","Data":"51ac3a9647c3c74361d383c5fa2d21ea3fda12eb820f194ad3468007b1c09439"} Jul 02 14:48:30.118388 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.118345 2581 scope.go:117] "RemoveContainer" containerID="7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed" Jul 02 14:48:30.119664 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.119635 2581 generic.go:356] "Generic (PLEG): container finished" podID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerID="74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256" exitCode=0 Jul 02 14:48:30.119778 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.119675 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" event={"ID":"54a22088-ebd6-43b6-bd9b-897d5f5eda7c","Type":"ContainerDied","Data":"74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256"} Jul 02 14:48:30.119778 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.119702 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" Jul 02 14:48:30.119778 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.119713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m" event={"ID":"54a22088-ebd6-43b6-bd9b-897d5f5eda7c","Type":"ContainerDied","Data":"75e37d77cde615203c9e0fe9986cd7e82527ff05835cb0fa941fb0df74a64f72"} Jul 02 14:48:30.127302 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.127090 2581 scope.go:117] "RemoveContainer" containerID="7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed" Jul 02 14:48:30.127543 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:48:30.127439 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed\": container with ID starting with 7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed not found: ID does not exist" containerID="7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed" Jul 02 14:48:30.127543 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.127478 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed"} err="failed to get container status \"7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed\": rpc error: code = NotFound desc = could not find container \"7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed\": container with ID starting with 7bf0e05e1882dfd65570e8e8cb1fdc52a2fec8a0c4c865a3bccd9fec095ba5ed not found: ID does not exist" Jul 02 14:48:30.127543 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.127498 2581 scope.go:117] "RemoveContainer" containerID="74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256" Jul 02 14:48:30.135328 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.135310 2581 scope.go:117] "RemoveContainer" containerID="74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256" Jul 02 14:48:30.135597 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:48:30.135580 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256\": container with ID starting with 74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256 not found: ID does not exist" containerID="74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256" Jul 02 14:48:30.135640 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.135606 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256"} err="failed to get container status \"74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256\": rpc error: code = NotFound desc = could not find container \"74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256\": container with ID starting with 74e0cbcfa1bcb29271aebb1e188e6ec803f74afc9435aeecfba1c48db97bd256 not found: ID does not exist" Jul 02 14:48:30.144442 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.144420 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m"] Jul 02 14:48:30.148826 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.148802 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-484ca-db8ddbcc6-fjb5m"] Jul 02 14:48:30.162705 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.162683 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99"] Jul 02 14:48:30.165246 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.165224 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3f32-d7476f4ff-9nl99"] Jul 02 14:48:30.284709 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.284678 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" path="/var/lib/kubelet/pods/54a22088-ebd6-43b6-bd9b-897d5f5eda7c/volumes" Jul 02 14:48:30.285023 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:48:30.285011 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" path="/var/lib/kubelet/pods/828bdff3-9b07-4549-bf67-d8c99187a2f9/volumes" Jul 02 14:49:09.489241 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489206 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92"] Jul 02 14:49:09.489649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489493 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" Jul 02 14:49:09.489649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489503 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" Jul 02 14:49:09.489649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489543 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" Jul 02 14:49:09.489649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489549 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" Jul 02 14:49:09.489649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489608 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="54a22088-ebd6-43b6-bd9b-897d5f5eda7c" containerName="splitter-graph-484ca" Jul 02 14:49:09.489649 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.489619 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="828bdff3-9b07-4549-bf67-d8c99187a2f9" containerName="sequence-graph-c3f32" Jul 02 14:49:09.492551 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.492533 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.495026 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.495003 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-cb06c-serving-cert\"" Jul 02 14:49:09.495173 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.495048 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 02 14:49:09.495173 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.495141 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-fn24b\"" Jul 02 14:49:09.496103 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.496081 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-cb06c-kube-rbac-proxy-sar-config\"" Jul 02 14:49:09.499266 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.499245 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:49:09.502381 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.502361 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92"] Jul 02 14:49:09.653770 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.653731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76f73dd7-f380-41ed-ab11-44c557a263a6-proxy-tls\") pod \"switch-graph-cb06c-565f58d8db-t6t92\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.653953 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.653782 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f73dd7-f380-41ed-ab11-44c557a263a6-openshift-service-ca-bundle\") pod \"switch-graph-cb06c-565f58d8db-t6t92\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.751101 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.751010 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:49:09.755072 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.755051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76f73dd7-f380-41ed-ab11-44c557a263a6-proxy-tls\") pod \"switch-graph-cb06c-565f58d8db-t6t92\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.755169 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.755089 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f73dd7-f380-41ed-ab11-44c557a263a6-openshift-service-ca-bundle\") pod \"switch-graph-cb06c-565f58d8db-t6t92\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.755741 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.755720 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f73dd7-f380-41ed-ab11-44c557a263a6-openshift-service-ca-bundle\") pod \"switch-graph-cb06c-565f58d8db-t6t92\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.757423 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.757401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76f73dd7-f380-41ed-ab11-44c557a263a6-proxy-tls\") pod \"switch-graph-cb06c-565f58d8db-t6t92\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.802942 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.802913 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:09.934271 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:09.934240 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92"] Jul 02 14:49:09.937257 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:49:09.937226 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f73dd7_f380_41ed_ab11_44c557a263a6.slice/crio-d62aaa3ca279cc2188dc2faac93dcaeeb6213bbb709bd005d13740e47da58f21 WatchSource:0}: Error finding container d62aaa3ca279cc2188dc2faac93dcaeeb6213bbb709bd005d13740e47da58f21: Status 404 returned error can't find the container with id d62aaa3ca279cc2188dc2faac93dcaeeb6213bbb709bd005d13740e47da58f21 Jul 02 14:49:10.004746 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.004687 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz"] Jul 02 14:49:10.007565 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.007549 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.010554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.009984 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-2963e-kube-rbac-proxy-sar-config\"" Jul 02 14:49:10.010554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.010006 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-2963e-serving-cert\"" Jul 02 14:49:10.010554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.010530 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:49:10.018201 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.018180 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz"] Jul 02 14:49:10.150391 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.150354 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:49:10.159396 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.159378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12591e02-4ff0-4088-a53f-c213b0997549-openshift-service-ca-bundle\") pod \"splitter-graph-2963e-85db8f6f9d-gw7pz\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.159466 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.159435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12591e02-4ff0-4088-a53f-c213b0997549-proxy-tls\") pod \"splitter-graph-2963e-85db8f6f9d-gw7pz\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.260025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.259931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12591e02-4ff0-4088-a53f-c213b0997549-proxy-tls\") pod \"splitter-graph-2963e-85db8f6f9d-gw7pz\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.260025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.259994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12591e02-4ff0-4088-a53f-c213b0997549-openshift-service-ca-bundle\") pod \"splitter-graph-2963e-85db8f6f9d-gw7pz\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.260651 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.260634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12591e02-4ff0-4088-a53f-c213b0997549-openshift-service-ca-bundle\") pod \"splitter-graph-2963e-85db8f6f9d-gw7pz\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.262317 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.262299 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12591e02-4ff0-4088-a53f-c213b0997549-proxy-tls\") pod \"splitter-graph-2963e-85db8f6f9d-gw7pz\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.287142 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.287106 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" event={"ID":"76f73dd7-f380-41ed-ab11-44c557a263a6","Type":"ContainerStarted","Data":"635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317"} Jul 02 14:49:10.287312 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.287166 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" event={"ID":"76f73dd7-f380-41ed-ab11-44c557a263a6","Type":"ContainerStarted","Data":"d62aaa3ca279cc2188dc2faac93dcaeeb6213bbb709bd005d13740e47da58f21"} Jul 02 14:49:10.287312 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.287189 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:10.304119 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.304073 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podStartSLOduration=1.304058347 podStartE2EDuration="1.304058347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:49:10.302993782 +0000 UTC m=+1954.424236177" watchObservedRunningTime="2026-07-02 14:49:10.304058347 +0000 UTC m=+1954.425300587" Jul 02 14:49:10.320145 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.320119 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:10.447127 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:10.447102 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz"] Jul 02 14:49:10.449655 ip-10-0-143-249 kubenswrapper[2581]: W0702 14:49:10.449632 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12591e02_4ff0_4088_a53f_c213b0997549.slice/crio-44fdc56c12d41c6b1c17712e5de1fc47b614e093207949a1f51a4f09e42f83c3 WatchSource:0}: Error finding container 44fdc56c12d41c6b1c17712e5de1fc47b614e093207949a1f51a4f09e42f83c3: Status 404 returned error can't find the container with id 44fdc56c12d41c6b1c17712e5de1fc47b614e093207949a1f51a4f09e42f83c3 Jul 02 14:49:11.292107 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:11.292060 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" event={"ID":"12591e02-4ff0-4088-a53f-c213b0997549","Type":"ContainerStarted","Data":"99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99"} Jul 02 14:49:11.292602 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:11.292121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" event={"ID":"12591e02-4ff0-4088-a53f-c213b0997549","Type":"ContainerStarted","Data":"44fdc56c12d41c6b1c17712e5de1fc47b614e093207949a1f51a4f09e42f83c3"} Jul 02 14:49:11.292602 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:11.292143 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:49:11.310437 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:11.310381 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podStartSLOduration=2.310365786 podStartE2EDuration="2.310365786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 14:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 14:49:11.309407807 +0000 UTC m=+1955.430650048" watchObservedRunningTime="2026-07-02 14:49:11.310365786 +0000 UTC m=+1955.431608025" Jul 02 14:49:16.297791 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:16.297765 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 14:49:17.300970 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:49:17.300938 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:50:19.468690 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:50:19.468655 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:51:33.981025 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:51:33.980983 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:51:36.386705 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:51:36.386678 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:51:36.391124 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:51:36.391104 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:55:45.011214 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:55:45.011116 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:55:45.070298 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:55:45.070265 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:55:55.557953 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:55:55.557914 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:56:36.408356 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:56:36.408326 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:56:36.413921 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:56:36.413902 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 14:57:24.630432 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:24.630400 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz"] Jul 02 14:57:24.630917 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:24.630622 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" containerID="cri-o://99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99" gracePeriod=30 Jul 02 14:57:24.662522 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:24.662493 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:57:25.795823 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:25.795788 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 14:57:27.299845 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:27.299803 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:57:32.299486 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:32.299448 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:57:37.299301 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:37.299263 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:57:37.299676 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:37.299364 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:57:42.299615 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:42.299572 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:57:47.299364 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:47.299326 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:57:52.300202 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:52.300141 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 14:57:54.770662 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.770638 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:57:54.801677 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.801648 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12591e02-4ff0-4088-a53f-c213b0997549-proxy-tls\") pod \"12591e02-4ff0-4088-a53f-c213b0997549\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " Jul 02 14:57:54.801838 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.801741 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12591e02-4ff0-4088-a53f-c213b0997549-openshift-service-ca-bundle\") pod \"12591e02-4ff0-4088-a53f-c213b0997549\" (UID: \"12591e02-4ff0-4088-a53f-c213b0997549\") " Jul 02 14:57:54.802139 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.802106 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12591e02-4ff0-4088-a53f-c213b0997549-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "12591e02-4ff0-4088-a53f-c213b0997549" (UID: "12591e02-4ff0-4088-a53f-c213b0997549"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 14:57:54.803745 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.803718 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12591e02-4ff0-4088-a53f-c213b0997549-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "12591e02-4ff0-4088-a53f-c213b0997549" (UID: "12591e02-4ff0-4088-a53f-c213b0997549"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 14:57:54.903064 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.902980 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12591e02-4ff0-4088-a53f-c213b0997549-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:57:54.903064 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:54.903011 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12591e02-4ff0-4088-a53f-c213b0997549-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 14:57:55.430297 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.430266 2581 generic.go:356] "Generic (PLEG): container finished" podID="12591e02-4ff0-4088-a53f-c213b0997549" containerID="99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99" exitCode=0 Jul 02 14:57:55.430554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.430311 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" Jul 02 14:57:55.430554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.430346 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" event={"ID":"12591e02-4ff0-4088-a53f-c213b0997549","Type":"ContainerDied","Data":"99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99"} Jul 02 14:57:55.430554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.430386 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz" event={"ID":"12591e02-4ff0-4088-a53f-c213b0997549","Type":"ContainerDied","Data":"44fdc56c12d41c6b1c17712e5de1fc47b614e093207949a1f51a4f09e42f83c3"} Jul 02 14:57:55.430554 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.430406 2581 scope.go:117] "RemoveContainer" containerID="99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99" Jul 02 14:57:55.438974 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.438954 2581 scope.go:117] "RemoveContainer" containerID="99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99" Jul 02 14:57:55.439229 ip-10-0-143-249 kubenswrapper[2581]: E0702 14:57:55.439212 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99\": container with ID starting with 99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99 not found: ID does not exist" containerID="99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99" Jul 02 14:57:55.439288 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.439237 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99"} err="failed to get container status \"99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99\": rpc error: code = NotFound desc = could not find container \"99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99\": container with ID starting with 99a19a81301b6b69ac5355dc7930d4f03689d024386404ba752f2bb39bc04a99 not found: ID does not exist" Jul 02 14:57:55.453768 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.453745 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz"] Jul 02 14:57:55.456192 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:55.456145 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-2963e-85db8f6f9d-gw7pz"] Jul 02 14:57:56.285112 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:57:56.285070 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12591e02-4ff0-4088-a53f-c213b0997549" path="/var/lib/kubelet/pods/12591e02-4ff0-4088-a53f-c213b0997549/volumes" Jul 02 14:58:13.426461 ip-10-0-143-249 kubenswrapper[2581]: I0702 14:58:13.426430 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:01:36.436892 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:01:36.436816 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 15:01:36.439416 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:01:36.437098 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 15:05:21.189845 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:21.189802 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:05:28.834307 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:28.834269 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92"] Jul 02 15:05:28.834883 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:28.834501 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" containerID="cri-o://635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317" gracePeriod=30 Jul 02 15:05:28.866095 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:28.866059 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:05:29.020823 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:29.020786 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:05:29.124666 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:29.124569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:05:29.984855 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:29.984814 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:05:30.084914 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:30.084883 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:05:31.296703 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:31.296661 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 15:05:36.296306 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:36.296268 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 15:05:41.296129 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:41.296091 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 15:05:41.296620 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:41.296230 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 15:05:45.964992 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:45.964963 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:46.296305 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:46.296221 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 15:05:46.760271 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:46.760242 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:47.571421 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:47.571395 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:48.332670 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:48.332643 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:49.113007 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:49.112974 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:49.892576 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:49.892545 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:50.688351 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:50.688321 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:51.296521 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:51.296482 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 15:05:51.476746 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:51.476713 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:52.248881 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:52.248839 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:53.042644 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:53.042618 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:53.838270 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:53.838237 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:54.668351 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:54.668317 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:55.458878 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:55.458854 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:56.296261 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:56.296229 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 15:05:56.348382 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:56.348353 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-cb06c-565f58d8db-t6t92_76f73dd7-f380-41ed-ab11-44c557a263a6/switch-graph-cb06c/0.log" Jul 02 15:05:58.980636 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:58.980614 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 15:05:59.107071 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.106989 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f73dd7-f380-41ed-ab11-44c557a263a6-openshift-service-ca-bundle\") pod \"76f73dd7-f380-41ed-ab11-44c557a263a6\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " Jul 02 15:05:59.107071 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.107044 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76f73dd7-f380-41ed-ab11-44c557a263a6-proxy-tls\") pod \"76f73dd7-f380-41ed-ab11-44c557a263a6\" (UID: \"76f73dd7-f380-41ed-ab11-44c557a263a6\") " Jul 02 15:05:59.107344 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.107322 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76f73dd7-f380-41ed-ab11-44c557a263a6-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "76f73dd7-f380-41ed-ab11-44c557a263a6" (UID: "76f73dd7-f380-41ed-ab11-44c557a263a6"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 15:05:59.108941 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.108920 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76f73dd7-f380-41ed-ab11-44c557a263a6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "76f73dd7-f380-41ed-ab11-44c557a263a6" (UID: "76f73dd7-f380-41ed-ab11-44c557a263a6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 15:05:59.208425 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.208386 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/76f73dd7-f380-41ed-ab11-44c557a263a6-openshift-service-ca-bundle\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 15:05:59.208425 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.208416 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76f73dd7-f380-41ed-ab11-44c557a263a6-proxy-tls\") on node \"ip-10-0-143-249.ec2.internal\" DevicePath \"\"" Jul 02 15:05:59.377273 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.377238 2581 generic.go:356] "Generic (PLEG): container finished" podID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerID="635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317" exitCode=0 Jul 02 15:05:59.377482 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.377295 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" Jul 02 15:05:59.377482 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.377315 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" event={"ID":"76f73dd7-f380-41ed-ab11-44c557a263a6","Type":"ContainerDied","Data":"635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317"} Jul 02 15:05:59.377482 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.377410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92" event={"ID":"76f73dd7-f380-41ed-ab11-44c557a263a6","Type":"ContainerDied","Data":"d62aaa3ca279cc2188dc2faac93dcaeeb6213bbb709bd005d13740e47da58f21"} Jul 02 15:05:59.377482 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.377444 2581 scope.go:117] "RemoveContainer" containerID="635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317" Jul 02 15:05:59.385539 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.385523 2581 scope.go:117] "RemoveContainer" containerID="635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317" Jul 02 15:05:59.385775 ip-10-0-143-249 kubenswrapper[2581]: E0702 15:05:59.385759 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317\": container with ID starting with 635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317 not found: ID does not exist" containerID="635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317" Jul 02 15:05:59.385822 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.385781 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317"} err="failed to get container status \"635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317\": rpc error: code = NotFound desc = could not find container \"635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317\": container with ID starting with 635ac179e5f7bc0a645a9390bef64167c4fd19293af47300fb72387bc951b317 not found: ID does not exist" Jul 02 15:05:59.398934 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.398910 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92"] Jul 02 15:05:59.405209 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:05:59.405189 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-cb06c-565f58d8db-t6t92"] Jul 02 15:06:00.285669 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:00.285635 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" path="/var/lib/kubelet/pods/76f73dd7-f380-41ed-ab11-44c557a263a6/volumes" Jul 02 15:06:01.122920 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:01.122890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:06:01.303578 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:01.303549 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:06:01.367935 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:01.367907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:06:05.077379 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:05.077350 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dbr7t_cdc7ec1e-5d10-4369-b9a3-0e0f77b38757/global-pull-secret-syncer/0.log" Jul 02 15:06:05.292111 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:05.292078 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" Jul 02 15:06:05.518958 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:05.518927 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-249.ec2.internal_6477d183aafb4ad9f6bb122000884dcf/haproxy/0.log" Jul 02 15:06:07.288184 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:07.288135 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:06:07.362948 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:07.362915 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:06:09.635393 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:09.635367 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t8tzl_08c9600e-0707-4326-8ffa-9d12fd349f54/node-exporter/0.log" Jul 02 15:06:09.680780 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:09.680755 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t8tzl_08c9600e-0707-4326-8ffa-9d12fd349f54/kube-rbac-proxy/0.log" Jul 02 15:06:09.716671 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:09.716647 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t8tzl_08c9600e-0707-4326-8ffa-9d12fd349f54/init-textfile/0.log" Jul 02 15:06:11.515581 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:11.515552 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qkws_9b41bd1d-1f33-40b9-95a0-2347fa038e15/konnectivity-agent/0.log" limit=1024 Jul 02 15:06:12.329567 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:12.329536 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/2.log" Jul 02 15:06:12.333482 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:12.333460 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6zxl_f98de5fa-8b99-409c-b217-eff19b9b536e/console-operator/3.log" Jul 02 15:06:12.785651 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:12.785621 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75458599c4-jtvql_faf7de05-72ac-4b13-8049-18d5c7fc892c/console/0.log" Jul 02 15:06:13.396019 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.395990 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-ck2wk_32cd5102-1e48-45fe-b380-10c49eeaa8cf/volume-data-source-validator/0.log" Jul 02 15:06:13.885062 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885031 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h"] Jul 02 15:06:13.885445 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885368 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" Jul 02 15:06:13.885445 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885381 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" Jul 02 15:06:13.885445 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885390 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" Jul 02 15:06:13.885445 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885396 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" Jul 02 15:06:13.885576 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885452 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="76f73dd7-f380-41ed-ab11-44c557a263a6" containerName="switch-graph-cb06c" Jul 02 15:06:13.885576 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.885460 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12591e02-4ff0-4088-a53f-c213b0997549" containerName="splitter-graph-2963e" Jul 02 15:06:13.889763 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.889743 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:13.894417 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.894397 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rvcf4\"/\"default-dockercfg-xczzd\"" Jul 02 15:06:13.894911 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.894891 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rvcf4\"/\"openshift-service-ca.crt\"" Jul 02 15:06:13.895766 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.895714 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rvcf4\"/\"kube-root-ca.crt\"" Jul 02 15:06:13.905193 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:13.905148 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h"] Jul 02 15:06:14.029415 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.029378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-sys\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.029609 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.029437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-podres\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.029609 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.029504 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmvj9\" (UniqueName: \"kubernetes.io/projected/fa224b3f-904d-41f2-a397-4ff8546bb8c0-kube-api-access-xmvj9\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.029609 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.029549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-proc\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.029609 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.029584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-lib-modules\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130631 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130598 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-proc\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130853 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-lib-modules\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130853 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130720 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-proc\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130853 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130758 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-sys\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130853 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-podres\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130853 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130814 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-lib-modules\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.130853 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130831 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmvj9\" (UniqueName: \"kubernetes.io/projected/fa224b3f-904d-41f2-a397-4ff8546bb8c0-kube-api-access-xmvj9\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.131053 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130855 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-sys\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.131053 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.130879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fa224b3f-904d-41f2-a397-4ff8546bb8c0-podres\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.152568 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.152507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmvj9\" (UniqueName: \"kubernetes.io/projected/fa224b3f-904d-41f2-a397-4ff8546bb8c0-kube-api-access-xmvj9\") pod \"perf-node-gather-daemonset-8zz9h\" (UID: \"fa224b3f-904d-41f2-a397-4ff8546bb8c0\") " pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.198923 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.198879 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.345012 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.344978 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h"] Jul 02 15:06:14.349020 ip-10-0-143-249 kubenswrapper[2581]: W0702 15:06:14.348992 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfa224b3f_904d_41f2_a397_4ff8546bb8c0.slice/crio-33e61dcdb5d0279c8c7087b7a69349cddaa3d82ff98f7e853bdcf3bb7a0047a9 WatchSource:0}: Error finding container 33e61dcdb5d0279c8c7087b7a69349cddaa3d82ff98f7e853bdcf3bb7a0047a9: Status 404 returned error can't find the container with id 33e61dcdb5d0279c8c7087b7a69349cddaa3d82ff98f7e853bdcf3bb7a0047a9 Jul 02 15:06:14.437280 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.437194 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" event={"ID":"fa224b3f-904d-41f2-a397-4ff8546bb8c0","Type":"ContainerStarted","Data":"751ea460eda2ce6ba0bf6fa0de9e729b60442fec8b878fcf802eb4d5a13fdd0a"} Jul 02 15:06:14.437280 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.437262 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:14.437490 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.437320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" event={"ID":"fa224b3f-904d-41f2-a397-4ff8546bb8c0","Type":"ContainerStarted","Data":"33e61dcdb5d0279c8c7087b7a69349cddaa3d82ff98f7e853bdcf3bb7a0047a9"} Jul 02 15:06:14.456491 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.456441 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" podStartSLOduration=1.4564257870000001 podStartE2EDuration="1.456425787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 15:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 15:06:14.454625093 +0000 UTC m=+2978.575867355" watchObservedRunningTime="2026-07-02 15:06:14.456425787 +0000 UTC m=+2978.577668027" Jul 02 15:06:14.590170 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.590119 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tgp6n_978d29b1-a9d0-44ee-95d2-11572d394c92/dns/0.log" Jul 02 15:06:14.615778 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.615743 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tgp6n_978d29b1-a9d0-44ee-95d2-11572d394c92/kube-rbac-proxy/0.log" Jul 02 15:06:14.640937 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:14.640913 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fsmwf_545cf0b4-dd6f-489f-8f52-3c697db25831/dns-node-resolver/0.log" Jul 02 15:06:15.206169 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:15.206135 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-b54fdc5c8-vzch2_0bbc3c98-b1d0-41df-a195-4fe50fb1ad67/registry/0.log" Jul 02 15:06:15.257778 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:15.257732 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kd2r7_2bffbb99-79da-4149-8564-b3b77a67615f/node-ca/0.log" Jul 02 15:06:16.459963 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:16.459930 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c9p6q_9a1b3fcc-b59c-480d-82eb-87a29f8cb07a/serve-healthcheck-canary/0.log" Jul 02 15:06:16.853362 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:16.853293 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-fsmx9_3e2045a4-0e5b-483b-9f85-43d1ab8b8d33/insights-operator/0.log" Jul 02 15:06:16.853720 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:16.853701 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-fsmx9_3e2045a4-0e5b-483b-9f85-43d1ab8b8d33/insights-operator/1.log" Jul 02 15:06:16.941779 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:16.941749 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h4l85_eee82ccd-bc7b-420e-83df-9b148b9eff9d/kube-rbac-proxy/0.log" Jul 02 15:06:16.963980 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:16.963951 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h4l85_eee82ccd-bc7b-420e-83df-9b148b9eff9d/exporter/0.log" Jul 02 15:06:16.986168 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:16.986127 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h4l85_eee82ccd-bc7b-420e-83df-9b148b9eff9d/extractor/0.log" Jul 02 15:06:19.181342 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:19.181312 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-tcdjh_e3869f4f-1b7b-44fc-b165-1dfcb925e876/manager/0.log" Jul 02 15:06:20.450815 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:20.450786 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rvcf4/perf-node-gather-daemonset-8zz9h" Jul 02 15:06:23.958687 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:23.958655 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-4dnzn_87a7e4f0-15dd-4ee3-bf66-9a0033fa496c/kube-storage-version-migrator-operator/1.log" Jul 02 15:06:23.959654 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:23.959636 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-4dnzn_87a7e4f0-15dd-4ee3-bf66-9a0033fa496c/kube-storage-version-migrator-operator/0.log" Jul 02 15:06:24.891377 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:24.891350 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9fsww_a8ec2efe-70d4-4636-9f1a-914333629928/kube-multus/0.log" Jul 02 15:06:24.918357 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:24.918330 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/kube-multus-additional-cni-plugins/0.log" Jul 02 15:06:24.941634 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:24.941611 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/egress-router-binary-copy/0.log" Jul 02 15:06:24.963328 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:24.963306 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/cni-plugins/0.log" Jul 02 15:06:24.986073 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:24.986035 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/bond-cni-plugin/0.log" Jul 02 15:06:25.009397 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:25.009373 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/routeoverride-cni/0.log" Jul 02 15:06:25.032912 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:25.032891 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/whereabouts-cni-bincopy/0.log" Jul 02 15:06:25.059536 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:25.059512 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2rxvc_c468cb3a-4a0d-48db-8aae-80b251657d8a/whereabouts-cni/0.log" Jul 02 15:06:25.469661 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:25.469639 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4mhz9_7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3/network-metrics-daemon/0.log" Jul 02 15:06:25.495583 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:25.495557 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4mhz9_7e05eeb2-39fa-4a9b-8d4e-22bc10dadfa3/kube-rbac-proxy/0.log" Jul 02 15:06:26.568457 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.568428 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/ovn-controller/0.log" Jul 02 15:06:26.595979 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.595949 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/ovn-acl-logging/0.log" Jul 02 15:06:26.649875 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.649841 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/kube-rbac-proxy-node/0.log" Jul 02 15:06:26.714749 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.714710 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/kube-rbac-proxy-ovn-metrics/0.log" Jul 02 15:06:26.777469 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.777444 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/northd/0.log" Jul 02 15:06:26.808875 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.808849 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/nbdb/0.log" Jul 02 15:06:26.832932 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.832846 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/sbdb/0.log" Jul 02 15:06:26.876847 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:26.876816 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-m7grd_c5630e44-468e-4982-b742-d7b34aea53d2/ovnkube-controller/0.log" Jul 02 15:06:28.112973 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:28.112943 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-8vhdd_43db889f-6d33-4460-a927-c974412e98dd/network-check-target-container/0.log" Jul 02 15:06:29.237946 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:29.237915 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qpqhq_5fdfa295-ce5b-40f8-bb82-71d6ec26a879/iptables-alerter/0.log" Jul 02 15:06:29.900962 ip-10-0-143-249 kubenswrapper[2581]: I0702 15:06:29.900934 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-c5vtl_15be550e-4122-4920-bc22-669d9b336248/tuned/0.log"