Jul 24 13:07:18.314299 ip-10-0-139-210 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 24 13:07:18.314309 ip-10-0-139-210 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 24 13:07:18.314318 ip-10-0-139-210 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 24 13:07:18.314560 ip-10-0-139-210 systemd[1]: Failed to start Kubernetes Kubelet. Jul 24 13:07:27.859323 ip-10-0-139-210 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 24 13:07:27.859338 ip-10-0-139-210 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ddc4ae27a89c428b8c9547648427178d -- Jul 24 13:08:56.518734 ip-10-0-139-210 systemd[1]: Starting Kubernetes Kubelet... Jul 24 13:08:56.918410 ip-10-0-139-210 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:08:56.918410 ip-10-0-139-210 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 13:08:56.918410 ip-10-0-139-210 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:08:56.918410 ip-10-0-139-210 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 13:08:56.918410 ip-10-0-139-210 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:08:56.919145 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.919052 2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 13:08:56.926392 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926364 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926394 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926402 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926406 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926410 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926413 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926416 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926419 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926422 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926425 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926428 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926430 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926433 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926435 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926438 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:08:56.926442 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926449 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926452 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926455 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926459 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926463 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926466 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926469 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926471 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926474 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926476 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926479 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926482 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926484 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926487 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926489 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926492 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926494 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926497 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926523 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:08:56.926853 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926540 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926544 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926547 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926550 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926554 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926557 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926560 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926563 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926565 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926568 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926571 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926573 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926576 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926579 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926581 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926584 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926587 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926589 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926592 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926595 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:08:56.927305 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926598 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926601 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926604 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926607 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926609 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926612 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926616 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926619 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926622 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926625 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926629 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926632 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926635 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926639 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926641 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926644 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926646 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926649 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926651 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926654 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:08:56.927811 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926657 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926659 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926662 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926664 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926667 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926671 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926674 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926676 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926679 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926681 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926684 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926686 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926689 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926691 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926694 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926697 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926699 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926701 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926704 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926707 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:08:56.928301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926710 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926713 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926716 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926718 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.926722 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927590 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927599 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927602 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927605 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927608 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927611 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927614 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927617 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927620 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927624 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927627 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927629 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927633 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927636 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927639 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:08:56.928821 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927641 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927644 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927646 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927649 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927652 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927655 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927658 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927661 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927664 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927666 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927669 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927671 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927677 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927680 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927685 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927688 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927691 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927693 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927696 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:08:56.929303 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927698 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927701 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927704 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927706 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927709 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927711 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927714 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927716 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927719 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927721 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927724 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927727 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927729 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927731 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927734 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927736 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927739 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927742 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927745 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927747 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:08:56.929880 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927750 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927752 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927755 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927757 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927759 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927762 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927765 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927768 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927771 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927773 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927776 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927778 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927781 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927784 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927786 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927789 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927791 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927794 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927796 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927799 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927802 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:08:56.930381 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927804 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927807 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927809 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927812 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927815 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927818 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927821 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927824 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927827 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927829 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927834 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927837 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927840 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927843 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927845 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927849 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927853 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927857 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:08:56.930922 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927862 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927866 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927869 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927871 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927874 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.927876 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928267 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928271 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928274 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928277 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928279 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928282 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928285 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928287 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928290 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928292 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928295 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928298 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928300 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928303 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:08:56.931361 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928306 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928309 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928312 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928317 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928319 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928322 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928324 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928327 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928329 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928332 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928334 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928337 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928340 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928344 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928346 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928349 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928351 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928354 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928356 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:08:56.931871 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928359 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928361 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928363 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928366 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928368 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928372 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928375 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928379 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928396 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928400 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928402 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928405 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928409 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928411 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928416 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928419 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928422 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928425 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928428 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:08:56.932371 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928431 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928434 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928436 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928440 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928442 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928445 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928451 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928454 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928457 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928460 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928463 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928466 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928469 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928471 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928474 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928477 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928479 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928482 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928485 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928487 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:08:56.932875 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928490 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928493 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928496 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928499 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928501 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928504 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928506 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928509 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928512 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928514 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928516 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928519 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928522 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928524 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928527 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928529 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928532 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928534 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928538 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928541 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:08:56.933351 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928544 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928547 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928549 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928552 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928554 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928556 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.928559 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928638 2558 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928645 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928651 2558 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928658 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928665 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928670 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928677 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928683 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928687 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928690 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928694 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928697 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928700 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928703 2558 flags.go:64] FLAG: --cgroup-root="" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928706 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928709 2558 flags.go:64] FLAG: --client-ca-file="" Jul 24 13:08:56.933881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928712 2558 flags.go:64] FLAG: --cloud-provider="external" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928715 2558 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928720 2558 flags.go:64] FLAG: --cluster-domain="" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928723 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928726 2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928730 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928733 2558 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928739 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928742 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928745 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928749 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928753 2558 flags.go:64] FLAG: --contention-profiling="false" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928756 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928759 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928762 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928764 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928769 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928772 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928775 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928778 2558 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928782 2558 flags.go:64] FLAG: --enable-server="true" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928785 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928789 2558 flags.go:64] FLAG: --event-burst="100" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928792 2558 flags.go:64] FLAG: --event-qps="50" Jul 24 13:08:56.934446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928795 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928798 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928801 2558 flags.go:64] FLAG: --eviction-hard="" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928805 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928808 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928811 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928814 2558 flags.go:64] FLAG: --eviction-soft="" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928822 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928825 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928829 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928832 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928835 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928838 2558 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928841 2558 flags.go:64] FLAG: --feature-gates="" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928845 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928849 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928852 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928856 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928859 2558 flags.go:64] FLAG: --healthz-port="10248" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928862 2558 flags.go:64] FLAG: --help="false" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928865 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-139-210.ec2.internal" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928868 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928872 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 13:08:56.935068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928875 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928878 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928881 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928884 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928887 2558 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928890 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928893 2558 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928896 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928899 2558 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928902 2558 flags.go:64] FLAG: --kube-reserved="" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928905 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928908 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928911 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928914 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928917 2558 flags.go:64] FLAG: --lock-file="" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928920 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928924 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928927 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928932 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928935 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928938 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928941 2558 flags.go:64] FLAG: --logging-format="text" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928943 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928947 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 13:08:56.935682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928952 2558 flags.go:64] FLAG: --manifest-url="" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928956 2558 flags.go:64] FLAG: --manifest-url-header="" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928962 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928967 2558 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928973 2558 flags.go:64] FLAG: --max-pods="110" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928978 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928982 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928986 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928990 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928994 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.928999 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929004 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929014 2558 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929018 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929021 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929024 2558 flags.go:64] FLAG: --pod-cidr="" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929027 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929033 2558 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929036 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929039 2558 flags.go:64] FLAG: --pods-per-core="0" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929042 2558 flags.go:64] FLAG: --port="10250" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929045 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929048 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab1c73ebd59fb746" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929052 2558 flags.go:64] FLAG: --qos-reserved="" Jul 24 13:08:56.936271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929056 2558 flags.go:64] FLAG: --read-only-port="10255" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929059 2558 flags.go:64] FLAG: --register-node="true" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929062 2558 flags.go:64] FLAG: --register-with-taints="" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929066 2558 flags.go:64] FLAG: --registry-burst="10" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929069 2558 flags.go:64] FLAG: --registry-qps="5" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929071 2558 flags.go:64] FLAG: --reserved-cpus="" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929074 2558 flags.go:64] FLAG: --reserved-memory="" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929078 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929081 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929086 2558 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929089 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929092 2558 flags.go:64] FLAG: --runonce="false" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929095 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929098 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929101 2558 flags.go:64] FLAG: --seccomp-default="false" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929105 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929108 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929111 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929114 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929117 2558 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929120 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929123 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929126 2558 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929129 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929132 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 13:08:56.936888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929135 2558 flags.go:64] FLAG: --system-cgroups="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929137 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929144 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929147 2558 flags.go:64] FLAG: --tls-cert-file="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929150 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929155 2558 flags.go:64] FLAG: --tls-min-version="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929157 2558 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929161 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929164 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929167 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929170 2558 flags.go:64] FLAG: --v="2" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929174 2558 flags.go:64] FLAG: --version="false" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929178 2558 flags.go:64] FLAG: --vmodule="" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929182 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929186 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929310 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929316 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929320 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929323 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929326 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929329 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929333 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:08:56.937614 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929337 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929340 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929343 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929346 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929349 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929351 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929354 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929357 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929359 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929362 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929364 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929367 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929369 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929372 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929374 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929377 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929379 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929398 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929402 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929404 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:08:56.938449 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929407 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929411 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929415 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929418 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929420 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929423 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929427 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929432 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929435 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929438 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929441 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929444 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929447 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929450 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929453 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929456 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929459 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929462 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929464 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:08:56.939231 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929467 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929469 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929472 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929475 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929477 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929480 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929483 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929485 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929488 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929491 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929495 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929497 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929500 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929503 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929505 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929508 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929511 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929513 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929516 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:08:56.940044 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929521 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929524 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929526 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929529 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929531 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929534 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929537 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929539 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929543 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929545 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929548 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929551 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929553 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929556 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929558 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929561 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929564 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929566 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929569 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929571 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:08:56.940623 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929574 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929576 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929579 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929583 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929586 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929589 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929592 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929594 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929597 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929599 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929602 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929605 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929609 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.929611 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.929617 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:08:56.941278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.936288 2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.936305 2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.936319 2558 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.936325 2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936397 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936404 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936408 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936410 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936414 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936416 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936419 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936423 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936426 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936429 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936432 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936435 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936437 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936440 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936442 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936445 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:08:56.941794 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936447 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936450 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936453 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936455 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936459 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936462 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936466 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936468 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936471 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936474 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936476 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936479 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936482 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936485 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936488 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936490 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936493 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936495 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936498 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936500 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:08:56.942342 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936503 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936506 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936511 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936514 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936518 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936521 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936524 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936527 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936530 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936532 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936535 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936538 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936541 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936543 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936546 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936548 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936551 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936553 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936556 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936558 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:08:56.942865 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936561 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936564 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936567 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936569 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936572 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936576 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936578 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936581 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936584 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936586 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936589 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936592 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936595 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936597 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936600 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936603 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936605 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936608 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936611 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936613 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:08:56.943412 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936616 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936618 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936621 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936624 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936626 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936629 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936631 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936634 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936638 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936641 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936644 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936646 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936650 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936654 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936657 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936659 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936663 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936665 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936668 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:08:56.943914 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936671 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936674 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936676 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936679 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.936684 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936803 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936809 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936812 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936815 2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936818 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936821 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936824 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936827 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936829 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936832 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:08:56.944411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936835 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936837 2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936840 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936842 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936844 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936847 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936850 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936852 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936855 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936857 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936860 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936862 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936864 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936867 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936870 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936872 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936875 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936878 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936881 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936883 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:08:56.944781 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936886 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936888 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936891 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936893 2558 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936896 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936899 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936901 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936904 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936907 2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936910 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936912 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936915 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936917 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936920 2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936923 2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936926 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936928 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936931 2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936933 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:08:56.945374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936936 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936940 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936944 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936947 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936950 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936953 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936956 2558 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936958 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936961 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936964 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936967 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936969 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936972 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936974 2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936977 2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936979 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936982 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936984 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936987 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:08:56.946134 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936989 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936992 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936994 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.936996 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937000 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937002 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937004 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937007 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937009 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937012 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937016 2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937019 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937022 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937026 2558 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937029 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937031 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937034 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937036 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937039 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:08:56.946683 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937041 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937045 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937047 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937050 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937053 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937055 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937058 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937060 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937062 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937065 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937067 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:56.937070 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.937075 2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.937206 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 13:08:56.947143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.940822 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 13:08:56.947548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.942202 2558 server.go:1019] "Starting client certificate rotation" Jul 24 13:08:56.947548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.942316 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:08:56.947548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.942352 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:08:56.947548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.945754 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:08:56.950296 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.950275 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:08:56.962811 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.962785 2558 log.go:25] "Validated CRI v1 runtime API" Jul 24 13:08:56.964889 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.964873 2558 log.go:25] "Validated CRI v1 image API" Jul 24 13:08:56.965740 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.965717 2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 13:08:56.968195 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.968168 2558 fs.go:135] Filesystem UUIDs: map[4ba8053b-0ff5-4489-9205-559b45cb8d50:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d0798abf-8af4-45a4-9a9e-a179c154c911:/dev/nvme0n1p4] Jul 24 13:08:56.968263 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.968194 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 13:08:56.973682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.973559 2558 manager.go:216] Machine: {Timestamp:2026-07-24 13:08:56.972484062 +0000 UTC m=+0.336408144 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100234 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25d2b3fed43986c044661c6afb86a9 SystemUUID:ec25d2b3-fed4-3986-c044-661c6afb86a9 BootID:ddc4ae27-a89c-428b-8c95-47648427178d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:20:67:c9:58:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:20:67:c9:58:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:48:f9:e1:6f:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 13:08:56.973682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.973671 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 13:08:56.973830 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.973773 2558 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 13:08:56.974909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.974879 2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 13:08:56.975049 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.974911 2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-210.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 13:08:56.975446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.975431 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:08:56.976044 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.976033 2558 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 13:08:56.976078 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.976047 2558 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 13:08:56.976078 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.976057 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:08:56.976775 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.976765 2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:08:56.977570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.977559 2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 13:08:56.979929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.979917 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:08:56.980054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980045 2558 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 13:08:56.980096 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980076 2558 kubelet.go:486] "Attempting to sync node with API server" Jul 24 13:08:56.980096 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980094 2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 13:08:56.980164 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980105 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 13:08:56.980164 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980113 2558 kubelet.go:398] "Adding apiserver pod source" Jul 24 13:08:56.980164 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980123 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 13:08:56.980617 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980601 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:08:56.980692 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.980622 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:08:56.983726 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.983705 2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 13:08:56.985725 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.985704 2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 13:08:56.985806 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.985740 2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 13:08:56.987108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987096 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987116 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987127 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987133 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987142 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987148 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987155 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 13:08:56.987162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987161 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 13:08:56.987378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987168 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 13:08:56.987378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987173 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 13:08:56.987378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987182 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 13:08:56.987378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.987187 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 13:08:56.988685 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.988674 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 13:08:56.988685 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.988685 2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 13:08:56.991408 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.991375 2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-210.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 13:08:56.991482 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:56.991434 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 13:08:56.991482 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:56.991462 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-210.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 13:08:56.992078 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.992065 2558 server.go:1268] "Started kubelet" Jul 24 13:08:56.992195 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.992148 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 13:08:56.992285 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.992205 2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 13:08:56.992338 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.992277 2558 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 13:08:56.993072 ip-10-0-139-210 systemd[1]: Started Kubernetes Kubelet. Jul 24 13:08:56.993890 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.993872 2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 13:08:56.994471 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:56.994449 2558 server.go:310] "Adding debug handlers to kubelet server" Jul 24 13:08:56.999458 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:56.999420 2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 13:08:57.000805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.000779 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 13:08:57.000805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.000788 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 13:08:57.000994 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.000932 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.001334 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.000163 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc339864a57 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:56.992033367 +0000 UTC m=+0.355957449,LastTimestamp:2026-07-24 13:08:56.992033367 +0000 UTC m=+0.355957449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.001456 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.001446 2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 13:08:57.001556 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.001542 2558 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 13:08:57.001600 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.001560 2558 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 13:08:57.001750 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.001734 2558 reconstruct.go:97] "Volume reconstruction finished" Jul 24 13:08:57.001750 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.001750 2558 reconciler.go:29] "Reconciler: start to sync state" Jul 24 13:08:57.003633 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.003617 2558 factory.go:55] Registering systemd factory Jul 24 13:08:57.003633 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.003636 2558 factory.go:223] Registration of the systemd container factory successfully Jul 24 13:08:57.003894 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.003880 2558 factory.go:153] Registering CRI-O factory Jul 24 13:08:57.003962 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.003898 2558 factory.go:223] Registration of the crio container factory successfully Jul 24 13:08:57.004026 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.003977 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 13:08:57.004091 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.004048 2558 factory.go:103] Registering Raw factory Jul 24 13:08:57.004091 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.004079 2558 manager.go:1196] Started watching for new ooms in manager Jul 24 13:08:57.004655 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.004641 2558 manager.go:349] Starting recovery of all containers Jul 24 13:08:57.006206 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.006176 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-210.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 13:08:57.006343 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.006319 2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 13:08:57.014006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.013976 2558 manager.go:354] Recovery completed Jul 24 13:08:57.021003 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.020986 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.022072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.022044 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fzjmf" Jul 24 13:08:57.023263 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.023239 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.023336 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.023284 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.023336 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.023303 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.023994 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.023979 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.024072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.023979 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 13:08:57.024072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.024019 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 13:08:57.024072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.024040 2558 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:08:57.024700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.024684 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.024787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.024714 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.024787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.024728 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.025212 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.025143 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b62d4e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023263975 +0000 UTC m=+0.387188063,LastTimestamp:2026-07-24 13:08:57.023263975 +0000 UTC m=+0.387188063,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.026020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.026006 2558 policy_none.go:49] "None policy: Start" Jul 24 13:08:57.026082 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.026027 2558 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 13:08:57.026082 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.026049 2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 13:08:57.026974 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.026961 2558 policy_none.go:47] "Start" Jul 24 13:08:57.029913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.029893 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fzjmf" Jul 24 13:08:57.035095 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.035001 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b6347ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023293357 +0000 UTC m=+0.387217441,LastTimestamp:2026-07-24 13:08:57.023293357 +0000 UTC m=+0.387217441,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.048031 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.047836 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b638a39 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023310393 +0000 UTC m=+0.387234480,LastTimestamp:2026-07-24 13:08:57.023310393 +0000 UTC m=+0.387234480,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.049990 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.049975 2558 manager.go:337] "Starting Device Plugin manager" Jul 24 13:08:57.050063 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.050010 2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 13:08:57.050063 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.050024 2558 server.go:87] "Starting device plugin registration server" Jul 24 13:08:57.050281 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.050269 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 13:08:57.050361 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.050282 2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 13:08:57.050424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.050369 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 13:08:57.050514 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.050499 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 13:08:57.050514 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.050514 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 13:08:57.051077 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.051057 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 13:08:57.051151 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.051114 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.058757 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.058670 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-210.ec2.internal.18c53bc33b62d4e7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b62d4e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023263975 +0000 UTC m=+0.387188063,LastTimestamp:2026-07-24 13:08:57.02469903 +0000 UTC m=+0.388623115,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.067172 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.067086 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-210.ec2.internal.18c53bc33b6347ad\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b6347ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023293357 +0000 UTC m=+0.387217441,LastTimestamp:2026-07-24 13:08:57.024720887 +0000 UTC m=+0.388644971,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.077862 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.077762 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-210.ec2.internal.18c53bc33b638a39\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b638a39 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023310393 +0000 UTC m=+0.387234480,LastTimestamp:2026-07-24 13:08:57.024732555 +0000 UTC m=+0.388656636,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.085782 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.085699 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33d1c7bb8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.052208056 +0000 UTC m=+0.416132128,LastTimestamp:2026-07-24 13:08:57.052208056 +0000 UTC m=+0.416132128,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.151055 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.151025 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.152049 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.152019 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.152141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.152065 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.152141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.152079 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.152141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.152106 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.164235 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.164137 2558 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-139-210.ec2.internal.18c53bc33b62d4e7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-210.ec2.internal.18c53bc33b62d4e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-210.ec2.internal,UID:ip-10-0-139-210.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-210.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-210.ec2.internal,},FirstTimestamp:2026-07-24 13:08:57.023263975 +0000 UTC m=+0.387188063,LastTimestamp:2026-07-24 13:08:57.152046153 +0000 UTC m=+0.515970238,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-210.ec2.internal,}" Jul 24 13:08:57.175179 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.175110 2558 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-139-210.ec2.internal\" not found" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.188165 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.188135 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 13:08:57.189508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.189492 2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 13:08:57.189589 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.189530 2558 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 13:08:57.189589 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.189551 2558 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 13:08:57.189589 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.189585 2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 13:08:57.192008 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.191985 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:08:57.213355 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.213320 2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-210.ec2.internal\" not found" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.290508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.290475 2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal"] Jul 24 13:08:57.290597 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.290561 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.291525 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.291508 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.291646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.291536 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.291646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.291546 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.292954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.292942 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.293066 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293054 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.293101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293083 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.293683 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293667 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.293756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293697 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.293756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293709 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.293756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293675 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.293863 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293761 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.293863 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.293780 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.294856 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.294841 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.294923 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.294876 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.295537 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.295517 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.295649 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.295544 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.295649 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.295558 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.314137 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.314105 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-210.ec2.internal\" not found" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.318674 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.318657 2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-210.ec2.internal\" not found" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.375853 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.375823 2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:08:57.376864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.376848 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:08:57.376956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.376878 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:08:57.376956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.376889 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:08:57.376956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.376914 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.387518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.387497 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.387580 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.387522 2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-210.ec2.internal\": node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.403267 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.403241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71e034ab1388ffeb81ee87fb6369df72-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal\" (UID: \"71e034ab1388ffeb81ee87fb6369df72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.403267 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.403267 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71e034ab1388ffeb81ee87fb6369df72-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal\" (UID: \"71e034ab1388ffeb81ee87fb6369df72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.403428 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.403285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb3099a2ddbd5f01479c1918b5f9b91d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-210.ec2.internal\" (UID: \"cb3099a2ddbd5f01479c1918b5f9b91d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.429089 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.429031 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.503905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.503870 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71e034ab1388ffeb81ee87fb6369df72-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal\" (UID: \"71e034ab1388ffeb81ee87fb6369df72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.504071 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.503920 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71e034ab1388ffeb81ee87fb6369df72-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal\" (UID: \"71e034ab1388ffeb81ee87fb6369df72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.504071 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.503929 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/71e034ab1388ffeb81ee87fb6369df72-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal\" (UID: \"71e034ab1388ffeb81ee87fb6369df72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.504071 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.503947 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb3099a2ddbd5f01479c1918b5f9b91d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-210.ec2.internal\" (UID: \"cb3099a2ddbd5f01479c1918b5f9b91d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.504071 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.503960 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71e034ab1388ffeb81ee87fb6369df72-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal\" (UID: \"71e034ab1388ffeb81ee87fb6369df72\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.504071 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.503976 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cb3099a2ddbd5f01479c1918b5f9b91d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-210.ec2.internal\" (UID: \"cb3099a2ddbd5f01479c1918b5f9b91d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.530031 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.529991 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.616564 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.616539 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.622179 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.622156 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" Jul 24 13:08:57.630846 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.630824 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.731513 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.731381 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.831898 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.831865 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.933600 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:57.933546 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:57.941271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.941238 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 13:08:57.941444 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:57.941424 2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:08:58.038681 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:58.038592 2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-210.ec2.internal\" not found" Jul 24 13:08:58.069738 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.069522 2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:08:58.115726 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.115697 2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:08:58.168895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.168841 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 13:03:57 +0000 UTC" deadline="2028-04-15 10:15:43.769913094 +0000 UTC" Jul 24 13:08:58.168895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.168888 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15141h6m45.601031959s" Jul 24 13:08:58.199988 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:58.199955 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e034ab1388ffeb81ee87fb6369df72.slice/crio-7d5a130af4a97eefe9d3183ad0fc0eab09e187493ae77cbef2fcf07651ce7a25 WatchSource:0}: Error finding container 7d5a130af4a97eefe9d3183ad0fc0eab09e187493ae77cbef2fcf07651ce7a25: Status 404 returned error can't find the container with id 7d5a130af4a97eefe9d3183ad0fc0eab09e187493ae77cbef2fcf07651ce7a25 Jul 24 13:08:58.200274 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:08:58.200247 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3099a2ddbd5f01479c1918b5f9b91d.slice/crio-c7789045f7bb9c37d2a87b3a6582bc5569d58b700f0a0c24adac756e98b3b240 WatchSource:0}: Error finding container c7789045f7bb9c37d2a87b3a6582bc5569d58b700f0a0c24adac756e98b3b240: Status 404 returned error can't find the container with id c7789045f7bb9c37d2a87b3a6582bc5569d58b700f0a0c24adac756e98b3b240 Jul 24 13:08:58.201443 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.201413 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" Jul 24 13:08:58.205059 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.205043 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:08:58.215098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.215082 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:08:58.216126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.216109 2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" Jul 24 13:08:58.222765 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.222747 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:08:58.422016 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.421928 2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:08:58.753054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.752965 2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:08:58.983500 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.983470 2558 apiserver.go:52] "Watching apiserver" Jul 24 13:08:58.991313 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.991273 2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 13:08:58.992584 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.992552 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-87xcs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal","openshift-multus/network-metrics-daemon-pnd68","openshift-network-diagnostics/network-check-target-sdkhm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw","openshift-cluster-node-tuning-operator/tuned-jw8n5","openshift-multus/multus-additional-cni-plugins-tj56n","openshift-multus/multus-k9vv9","openshift-network-operator/iptables-alerter-bz7bk","openshift-ovn-kubernetes/ovnkube-node-2bvgd","kube-system/global-pull-secret-syncer-8jsc6","kube-system/konnectivity-agent-4nd5b","kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal","openshift-dns/node-resolver-zm4hb"] Jul 24 13:08:58.994837 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.994392 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:58.995779 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.995659 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:58.995779 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:58.995737 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:08:58.997092 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.997066 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:08:58.997197 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:58.997153 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:08:58.997681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.997618 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 13:08:58.997873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.997858 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 13:08:58.998558 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.998067 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 13:08:58.998558 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:58.998153 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z956h\"" Jul 24 13:08:59.001880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.001363 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.002847 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.002170 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 13:08:59.002847 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.002801 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.004084 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.004035 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 13:08:59.004257 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.004241 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 13:08:59.004663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.004641 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7svfn\"" Jul 24 13:08:59.005051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.004847 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.005051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.005021 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:08:59.005297 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.005277 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fk6cv\"" Jul 24 13:08:59.005615 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.005593 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 13:08:59.005879 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.005860 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 13:08:59.007576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.007556 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.008522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.008173 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 13:08:59.008522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.008185 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2w8tr\"" Jul 24 13:08:59.008522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.008258 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 13:08:59.008522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.008402 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 13:08:59.008522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.008444 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 13:08:59.008824 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.008794 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 13:08:59.009593 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.009564 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.010048 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010029 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 13:08:59.010441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010420 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qtmgd\"" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010868 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-tuned\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010901 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-kubelet-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010929 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-etc-selinux\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010953 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-sys-fs\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.010977 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-kubernetes\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011001 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-socket-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011030 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-device-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011054 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysctl-d\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbzpd\" (UniqueName: \"kubernetes.io/projected/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-kube-api-access-nbzpd\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011144 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-sys\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011174 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-host\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011201 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-host\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011260 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npn26\" (UniqueName: \"kubernetes.io/projected/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-kube-api-access-npn26\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011274 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011289 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-lib-modules\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011320 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-serviceca\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011350 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011379 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-modprobe-d\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011432 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-run\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011501 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c64l\" (UniqueName: \"kubernetes.io/projected/46b5d990-b4de-47e1-9607-4dcc723020ee-kube-api-access-6c64l\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011534 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011563 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-registration-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.011612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-var-lib-kubelet\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.012379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011631 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b5d990-b4de-47e1-9607-4dcc723020ee-tmp\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.012379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011672 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvqb\" (UniqueName: \"kubernetes.io/projected/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-kube-api-access-dxvqb\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.012379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011702 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysconfig\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.012379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysctl-conf\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.012379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.011789 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-systemd\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.013363 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.013322 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.013443 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.013399 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:08:59.014143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.014119 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 13:08:59.014921 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.014904 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:08:59.015837 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.015819 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 13:08:59.016406 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.016389 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 13:08:59.016941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.016924 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hwmw9\"" Jul 24 13:08:59.017809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.017791 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 13:08:59.018163 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.018145 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 13:08:59.018630 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.018612 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 13:08:59.019694 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.019216 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 13:08:59.019694 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.019612 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 13:08:59.022377 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.019975 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nw4tq\"" Jul 24 13:08:59.022377 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.020594 2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:08:59.023943 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.023311 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.027905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.027875 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.028431 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.028362 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d4szw\"" Jul 24 13:08:59.030720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.030425 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 13:08:59.030720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.030509 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 13:08:59.031049 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.031027 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 13:08:59.031201 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.031183 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 13:08:59.031833 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.031812 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q579w\"" Jul 24 13:08:59.041246 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.041139 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-j49pj" Jul 24 13:08:59.048954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.048879 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-j49pj" Jul 24 13:08:59.102596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.102524 2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 13:08:59.113003 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.112967 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysctl-conf\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113019 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-systemd\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113057 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-run-ovn-kubernetes\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.113154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113087 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35bcce45-7c4a-4313-b746-8d124094ebc6-host-slash\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.113154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113120 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-kubelet-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.113154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113148 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-sys-fs\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113174 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-kubernetes\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113210 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-os-release\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113241 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113269 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-node-log\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-log-socket\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113333 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-env-overrides\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113365 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysctl-d\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113391 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-system-cni-dir\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113447 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lqjq\" (UniqueName: \"kubernetes.io/projected/a15b19fb-dfbd-44af-8773-abe59fece774-kube-api-access-7lqjq\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113470 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysctl-conf\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113476 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-systemd-units\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113578 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-systemd\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113582 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-kubernetes\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113597 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-sys-fs\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113615 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-kubelet-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113665 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebd3b024-b11e-4db8-94c9-4344ba779060-cni-binary-copy\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113727 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-host\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113742 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysctl-d\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113779 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-host\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113767 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-etc-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.113834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113816 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-ovn\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113845 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-cni-netd\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113890 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/24987975-7733-4719-aab2-ca011b0f4e86-ovn-node-metrics-cert\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113916 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/077ed00b-9339-4528-948b-daf1bebab5fc-hosts-file\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113942 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-etc-kubernetes\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113971 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-host\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.113998 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114023 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/077ed00b-9339-4528-948b-daf1bebab5fc-tmp-dir\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114031 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-host\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114052 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-run\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114079 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c64l\" (UniqueName: \"kubernetes.io/projected/46b5d990-b4de-47e1-9607-4dcc723020ee-kube-api-access-6c64l\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114102 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4971787d-2c92-40a4-94d9-6ab28883e02b-kubelet-config\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114132 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b5d990-b4de-47e1-9607-4dcc723020ee-tmp\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114154 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-systemd\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114180 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-ovnkube-script-lib\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114189 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-run\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114210 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-cni-bin\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.114424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114239 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05df2885-2fde-48f1-aaf8-1a820a0d2f2a-konnectivity-ca\") pod \"konnectivity-agent-4nd5b\" (UID: \"05df2885-2fde-48f1-aaf8-1a820a0d2f2a\") " pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114286 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114332 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-ovnkube-config\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114363 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-cnibin\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114399 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysconfig\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114465 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-cni-binary-copy\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-kubelet\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114523 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-tuned\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114521 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114539 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-sysconfig\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114550 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-etc-selinux\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114582 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114635 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-slash\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114668 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-cni-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114709 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-os-release\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114738 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-etc-selinux\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114742 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-conf-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.115278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114785 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-socket-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114814 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-device-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114895 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z6m6\" (UniqueName: \"kubernetes.io/projected/077ed00b-9339-4528-948b-daf1bebab5fc-kube-api-access-9z6m6\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114924 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-system-cni-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114950 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqs7d\" (UniqueName: \"kubernetes.io/projected/ebd3b024-b11e-4db8-94c9-4344ba779060-kube-api-access-nqs7d\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114982 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-socket-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.114983 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbzpd\" (UniqueName: \"kubernetes.io/projected/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-kube-api-access-nbzpd\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115025 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-sys\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115051 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg7hg\" (UniqueName: \"kubernetes.io/projected/24987975-7733-4719-aab2-ca011b0f4e86-kube-api-access-wg7hg\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115113 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-daemon-config\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115140 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m5cj\" (UniqueName: \"kubernetes.io/projected/35bcce45-7c4a-4313-b746-8d124094ebc6-kube-api-access-5m5cj\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115169 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npn26\" (UniqueName: \"kubernetes.io/projected/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-kube-api-access-npn26\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-k8s-cni-cncf-io\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115222 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-kubelet\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-multus-certs\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115261 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-device-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116124 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115272 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35bcce45-7c4a-4313-b746-8d124094ebc6-iptables-alerter-script\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115300 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-cni-bin\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115325 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-lib-modules\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115355 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-serviceca\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115380 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115406 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-modprobe-d\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115458 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05df2885-2fde-48f1-aaf8-1a820a0d2f2a-agent-certs\") pod \"konnectivity-agent-4nd5b\" (UID: \"05df2885-2fde-48f1-aaf8-1a820a0d2f2a\") " pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115486 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-cnibin\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115511 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-socket-dir-parent\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115535 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-sys\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115541 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115568 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-registration-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115627 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-registration-dir\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.115718 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.115787 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:08:59.615763866 +0000 UTC m=+2.979687971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.115891 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-modprobe-d\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116028 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-var-lib-kubelet\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.116873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116073 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-serviceca\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-var-lib-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116096 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-lib-modules\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116110 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116142 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4971787d-2c92-40a4-94d9-6ab28883e02b-dbus\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116148 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46b5d990-b4de-47e1-9607-4dcc723020ee-var-lib-kubelet\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116171 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-cni-multus\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116198 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-hostroot\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116226 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvqb\" (UniqueName: \"kubernetes.io/projected/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-kube-api-access-dxvqb\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116245 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-run-netns\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.117739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.116261 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-netns\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.120303 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.120278 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b5d990-b4de-47e1-9607-4dcc723020ee-tmp\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.120386 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.120303 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/46b5d990-b4de-47e1-9607-4dcc723020ee-etc-tuned\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.125910 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.125881 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:08:59.126019 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.125926 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:08:59.126019 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.125941 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:08:59.126019 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.126003 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:08:59.625983896 +0000 UTC m=+2.989907993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:08:59.128639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.128608 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbzpd\" (UniqueName: \"kubernetes.io/projected/8ceaada0-e85b-46d4-b0a4-7b3088dbc031-kube-api-access-nbzpd\") pod \"aws-ebs-csi-driver-node-56bhw\" (UID: \"8ceaada0-e85b-46d4-b0a4-7b3088dbc031\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.131694 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.131673 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c64l\" (UniqueName: \"kubernetes.io/projected/46b5d990-b4de-47e1-9607-4dcc723020ee-kube-api-access-6c64l\") pod \"tuned-jw8n5\" (UID: \"46b5d990-b4de-47e1-9607-4dcc723020ee\") " pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.133353 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.133328 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npn26\" (UniqueName: \"kubernetes.io/projected/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-kube-api-access-npn26\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:59.133455 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.133416 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvqb\" (UniqueName: \"kubernetes.io/projected/595f165a-81a3-4662-9fb1-6e5e7c7dacb9-kube-api-access-dxvqb\") pod \"node-ca-87xcs\" (UID: \"595f165a-81a3-4662-9fb1-6e5e7c7dacb9\") " pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.195197 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.195135 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" event={"ID":"cb3099a2ddbd5f01479c1918b5f9b91d","Type":"ContainerStarted","Data":"c7789045f7bb9c37d2a87b3a6582bc5569d58b700f0a0c24adac756e98b3b240"} Jul 24 13:08:59.196731 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.196698 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" event={"ID":"71e034ab1388ffeb81ee87fb6369df72","Type":"ContainerStarted","Data":"7d5a130af4a97eefe9d3183ad0fc0eab09e187493ae77cbef2fcf07651ce7a25"} Jul 24 13:08:59.216665 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216622 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4971787d-2c92-40a4-94d9-6ab28883e02b-kubelet-config\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.216665 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216666 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-systemd\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.216888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216690 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-ovnkube-script-lib\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.216888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216712 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-cni-bin\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.216888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216758 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-cni-bin\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.216888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216760 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4971787d-2c92-40a4-94d9-6ab28883e02b-kubelet-config\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.216888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216836 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05df2885-2fde-48f1-aaf8-1a820a0d2f2a-konnectivity-ca\") pod \"konnectivity-agent-4nd5b\" (UID: \"05df2885-2fde-48f1-aaf8-1a820a0d2f2a\") " pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.216888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216838 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-systemd\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216887 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216930 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216932 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-ovnkube-config\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.216970 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-cnibin\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217003 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-cni-binary-copy\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217039 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-kubelet\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217071 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-cnibin\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217081 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.217132 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217119 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-slash\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217149 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-cni-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217176 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-os-release\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217203 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-conf-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217236 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217236 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z6m6\" (UniqueName: \"kubernetes.io/projected/077ed00b-9339-4528-948b-daf1bebab5fc-kube-api-access-9z6m6\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217278 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-system-cni-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217304 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqs7d\" (UniqueName: \"kubernetes.io/projected/ebd3b024-b11e-4db8-94c9-4344ba779060-kube-api-access-nqs7d\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217350 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg7hg\" (UniqueName: \"kubernetes.io/projected/24987975-7733-4719-aab2-ca011b0f4e86-kube-api-access-wg7hg\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-daemon-config\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217387 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-ovnkube-script-lib\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217398 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m5cj\" (UniqueName: \"kubernetes.io/projected/35bcce45-7c4a-4313-b746-8d124094ebc6-kube-api-access-5m5cj\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217410 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-cni-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217441 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-k8s-cni-cncf-io\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217463 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-slash\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217464 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-kubelet\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217498 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-os-release\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.217566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217352 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/05df2885-2fde-48f1-aaf8-1a820a0d2f2a-konnectivity-ca\") pod \"konnectivity-agent-4nd5b\" (UID: \"05df2885-2fde-48f1-aaf8-1a820a0d2f2a\") " pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217588 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-kubelet\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217640 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-kubelet\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217695 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-multus-certs\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217740 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-multus-certs\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217759 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-conf-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217767 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-ovnkube-config\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.217973 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35bcce45-7c4a-4313-b746-8d124094ebc6-iptables-alerter-script\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218004 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-cni-bin\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218037 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-k8s-cni-cncf-io\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218063 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05df2885-2fde-48f1-aaf8-1a820a0d2f2a-agent-certs\") pod \"konnectivity-agent-4nd5b\" (UID: \"05df2885-2fde-48f1-aaf8-1a820a0d2f2a\") " pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218092 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-cnibin\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218063 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-system-cni-dir\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218135 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-cni-binary-copy\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218126 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-socket-dir-parent\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218174 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-socket-dir-parent\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-var-lib-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.218359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218262 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218295 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-cni-bin\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218319 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218345 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4971787d-2c92-40a4-94d9-6ab28883e02b-dbus\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218387 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-cni-multus\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218433 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-hostroot\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218458 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-run-netns\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218464 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4971787d-2c92-40a4-94d9-6ab28883e02b-dbus\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218471 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ebd3b024-b11e-4db8-94c9-4344ba779060-multus-daemon-config\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218489 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-var-lib-cni-multus\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218482 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-netns\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218509 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-hostroot\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218320 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-var-lib-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218551 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-run-netns\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218557 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-host-run-netns\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218558 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-cnibin\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218586 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-run-ovn-kubernetes\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218612 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35bcce45-7c4a-4313-b746-8d124094ebc6-iptables-alerter-script\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.219216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218614 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-run-ovn-kubernetes\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218634 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35bcce45-7c4a-4313-b746-8d124094ebc6-host-slash\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218701 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35bcce45-7c4a-4313-b746-8d124094ebc6-host-slash\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218730 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-os-release\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218759 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218786 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-node-log\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218799 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-os-release\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218810 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-log-socket\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218852 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-log-socket\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218876 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-node-log\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218882 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-env-overrides\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218918 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-system-cni-dir\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218943 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lqjq\" (UniqueName: \"kubernetes.io/projected/a15b19fb-dfbd-44af-8773-abe59fece774-kube-api-access-7lqjq\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218969 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-systemd-units\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218987 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a15b19fb-dfbd-44af-8773-abe59fece774-system-cni-dir\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.218994 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219025 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebd3b024-b11e-4db8-94c9-4344ba779060-cni-binary-copy\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219057 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-etc-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.219906 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.219078 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219127 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-etc-openvswitch\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219081 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-ovn\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219169 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-cni-netd\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219205 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/24987975-7733-4719-aab2-ca011b0f4e86-ovn-node-metrics-cert\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.219249 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:08:59.719229296 +0000 UTC m=+3.083153382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219276 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/077ed00b-9339-4528-948b-daf1bebab5fc-hosts-file\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-etc-kubernetes\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219077 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-systemd-units\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219295 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219381 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/077ed00b-9339-4528-948b-daf1bebab5fc-hosts-file\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219366 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/24987975-7733-4719-aab2-ca011b0f4e86-env-overrides\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219412 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219442 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ebd3b024-b11e-4db8-94c9-4344ba779060-etc-kubernetes\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219472 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/077ed00b-9339-4528-948b-daf1bebab5fc-tmp-dir\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219482 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-host-cni-netd\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219521 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/24987975-7733-4719-aab2-ca011b0f4e86-run-ovn\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.220491 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219666 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebd3b024-b11e-4db8-94c9-4344ba779060-cni-binary-copy\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.221024 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219787 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/077ed00b-9339-4528-948b-daf1bebab5fc-tmp-dir\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.221024 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.219959 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a15b19fb-dfbd-44af-8773-abe59fece774-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.221869 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.221850 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/05df2885-2fde-48f1-aaf8-1a820a0d2f2a-agent-certs\") pod \"konnectivity-agent-4nd5b\" (UID: \"05df2885-2fde-48f1-aaf8-1a820a0d2f2a\") " pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.221976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.221946 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/24987975-7733-4719-aab2-ca011b0f4e86-ovn-node-metrics-cert\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.229781 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.229755 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m5cj\" (UniqueName: \"kubernetes.io/projected/35bcce45-7c4a-4313-b746-8d124094ebc6-kube-api-access-5m5cj\") pod \"iptables-alerter-bz7bk\" (UID: \"35bcce45-7c4a-4313-b746-8d124094ebc6\") " pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.230055 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.230024 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z6m6\" (UniqueName: \"kubernetes.io/projected/077ed00b-9339-4528-948b-daf1bebab5fc-kube-api-access-9z6m6\") pod \"node-resolver-zm4hb\" (UID: \"077ed00b-9339-4528-948b-daf1bebab5fc\") " pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.230175 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.230026 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqs7d\" (UniqueName: \"kubernetes.io/projected/ebd3b024-b11e-4db8-94c9-4344ba779060-kube-api-access-nqs7d\") pod \"multus-k9vv9\" (UID: \"ebd3b024-b11e-4db8-94c9-4344ba779060\") " pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.230244 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.230187 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg7hg\" (UniqueName: \"kubernetes.io/projected/24987975-7733-4719-aab2-ca011b0f4e86-kube-api-access-wg7hg\") pod \"ovnkube-node-2bvgd\" (UID: \"24987975-7733-4719-aab2-ca011b0f4e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.230445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.230428 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lqjq\" (UniqueName: \"kubernetes.io/projected/a15b19fb-dfbd-44af-8773-abe59fece774-kube-api-access-7lqjq\") pod \"multus-additional-cni-plugins-tj56n\" (UID: \"a15b19fb-dfbd-44af-8773-abe59fece774\") " pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.312058 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.311962 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87xcs" Jul 24 13:08:59.329280 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.329241 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" Jul 24 13:08:59.337928 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.337887 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" Jul 24 13:08:59.343219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.343187 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tj56n" Jul 24 13:08:59.352102 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.352062 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k9vv9" Jul 24 13:08:59.361986 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.361955 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bz7bk" Jul 24 13:08:59.369988 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.369964 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:08:59.380155 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.380127 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:08:59.386040 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.386011 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zm4hb" Jul 24 13:08:59.622471 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.622350 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:08:59.622729 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.622549 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:08:59.622729 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.622630 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:09:00.62260803 +0000 UTC m=+3.986532100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:08:59.723097 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.723058 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:08:59.723097 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:08:59.723109 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:08:59.723352 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.723238 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:08:59.723352 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.723284 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:08:59.723352 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.723319 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:08:59.723352 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.723335 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:08:59.723572 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.723298 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:09:00.723280941 +0000 UTC m=+4.087205012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:08:59.723572 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:08:59.723433 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:00.723403496 +0000 UTC m=+4.087327579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:00.052518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:00.052424 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:03:59 +0000 UTC" deadline="2027-12-26 03:08:21.189738521 +0000 UTC" Jul 24 13:09:00.052518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:00.052465 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12469h59m21.137279676s" Jul 24 13:09:00.190516 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:00.190483 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:00.190716 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.190625 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:00.264877 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.264847 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ceaada0_e85b_46d4_b0a4_7b3088dbc031.slice/crio-b6d05683ffadc4b4fb2adddde3c5d152ee4a9fc4d4598978d4a8922dd797acda WatchSource:0}: Error finding container b6d05683ffadc4b4fb2adddde3c5d152ee4a9fc4d4598978d4a8922dd797acda: Status 404 returned error can't find the container with id b6d05683ffadc4b4fb2adddde3c5d152ee4a9fc4d4598978d4a8922dd797acda Jul 24 13:09:00.267625 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.267552 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35bcce45_7c4a_4313_b746_8d124094ebc6.slice/crio-93d50954aa5fb7b899b195a74eae87a0c1fabdad0155c9dd16bca396dc22f134 WatchSource:0}: Error finding container 93d50954aa5fb7b899b195a74eae87a0c1fabdad0155c9dd16bca396dc22f134: Status 404 returned error can't find the container with id 93d50954aa5fb7b899b195a74eae87a0c1fabdad0155c9dd16bca396dc22f134 Jul 24 13:09:00.268374 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.268220 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd3b024_b11e_4db8_94c9_4344ba779060.slice/crio-23a2e90c667581cd3548e29eaba52e9d620d3fbdac5705cf93136c1caa117159 WatchSource:0}: Error finding container 23a2e90c667581cd3548e29eaba52e9d620d3fbdac5705cf93136c1caa117159: Status 404 returned error can't find the container with id 23a2e90c667581cd3548e29eaba52e9d620d3fbdac5705cf93136c1caa117159 Jul 24 13:09:00.270270 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.269925 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595f165a_81a3_4662_9fb1_6e5e7c7dacb9.slice/crio-b7a1c3b124144fa72789d6b90f3a2ad394e64039972e3e9d7644709c0909c88f WatchSource:0}: Error finding container b7a1c3b124144fa72789d6b90f3a2ad394e64039972e3e9d7644709c0909c88f: Status 404 returned error can't find the container with id b7a1c3b124144fa72789d6b90f3a2ad394e64039972e3e9d7644709c0909c88f Jul 24 13:09:00.272401 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.272206 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15b19fb_dfbd_44af_8773_abe59fece774.slice/crio-f3c520b9a5858b23f5d35b42e37730da7c05a1ff57f8b0a1ff07049796a4251a WatchSource:0}: Error finding container f3c520b9a5858b23f5d35b42e37730da7c05a1ff57f8b0a1ff07049796a4251a: Status 404 returned error can't find the container with id f3c520b9a5858b23f5d35b42e37730da7c05a1ff57f8b0a1ff07049796a4251a Jul 24 13:09:00.273461 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.273373 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b5d990_b4de_47e1_9607_4dcc723020ee.slice/crio-ba7f9d1d325508206626d1b5fd9c93a88fce9458fe64f007e6deffaa7c687fb9 WatchSource:0}: Error finding container ba7f9d1d325508206626d1b5fd9c93a88fce9458fe64f007e6deffaa7c687fb9: Status 404 returned error can't find the container with id ba7f9d1d325508206626d1b5fd9c93a88fce9458fe64f007e6deffaa7c687fb9 Jul 24 13:09:00.274376 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.274346 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24987975_7733_4719_aab2_ca011b0f4e86.slice/crio-e84227ed8c29e563e7d20005c9a171455c0927e9aa1fae86b567441f29530f05 WatchSource:0}: Error finding container e84227ed8c29e563e7d20005c9a171455c0927e9aa1fae86b567441f29530f05: Status 404 returned error can't find the container with id e84227ed8c29e563e7d20005c9a171455c0927e9aa1fae86b567441f29530f05 Jul 24 13:09:00.275444 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.275429 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05df2885_2fde_48f1_aaf8_1a820a0d2f2a.slice/crio-3f7977db1f4dd780de6962b4c12ab8acd70640132049315b82f57e325c467397 WatchSource:0}: Error finding container 3f7977db1f4dd780de6962b4c12ab8acd70640132049315b82f57e325c467397: Status 404 returned error can't find the container with id 3f7977db1f4dd780de6962b4c12ab8acd70640132049315b82f57e325c467397 Jul 24 13:09:00.277301 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:00.276816 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077ed00b_9339_4528_948b_daf1bebab5fc.slice/crio-495fb3af935e510c14097e40b9cec64af0a5c09f536a50c6474c12850b70d524 WatchSource:0}: Error finding container 495fb3af935e510c14097e40b9cec64af0a5c09f536a50c6474c12850b70d524: Status 404 returned error can't find the container with id 495fb3af935e510c14097e40b9cec64af0a5c09f536a50c6474c12850b70d524 Jul 24 13:09:00.629998 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:00.629700 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:00.630291 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.629861 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:00.630291 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.630072 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:09:02.630053124 +0000 UTC m=+5.993977198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:00.731418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:00.731369 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:00.731607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:00.731458 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:00.731607 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.731574 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:00.731702 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.731635 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:09:02.731621093 +0000 UTC m=+6.095545164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:00.731702 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.731572 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:00.731702 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.731698 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:00.731879 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.731714 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:00.731879 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:00.731786 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:02.731764447 +0000 UTC m=+6.095688533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:01.053283 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.053127 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:03:59 +0000 UTC" deadline="2028-03-14 15:17:46.151577805 +0000 UTC" Jul 24 13:09:01.053283 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.053198 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14378h8m45.098384427s" Jul 24 13:09:01.193787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.192981 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:01.193787 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:01.193127 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:01.193787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.193605 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:01.193787 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:01.193724 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:01.213057 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.213013 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" event={"ID":"cb3099a2ddbd5f01479c1918b5f9b91d","Type":"ContainerStarted","Data":"66cfdd9a4470e98124e0bd1af9d7e8f1bd2df1d9971837959cd2065e618a075d"} Jul 24 13:09:01.224919 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.224877 2558 generic.go:356] "Generic (PLEG): container finished" podID="71e034ab1388ffeb81ee87fb6369df72" containerID="7f0f3b440c57549ff8f539295a74738d395be99a749844378d5a52dd3459aa93" exitCode=0 Jul 24 13:09:01.225332 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.225164 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" event={"ID":"71e034ab1388ffeb81ee87fb6369df72","Type":"ContainerDied","Data":"7f0f3b440c57549ff8f539295a74738d395be99a749844378d5a52dd3459aa93"} Jul 24 13:09:01.230206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.229881 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-210.ec2.internal" podStartSLOduration=3.229860213 podStartE2EDuration="3.229860213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:01.229478994 +0000 UTC m=+4.593403081" watchObservedRunningTime="2026-07-24 13:09:01.229860213 +0000 UTC m=+4.593784315" Jul 24 13:09:01.232241 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.232175 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zm4hb" event={"ID":"077ed00b-9339-4528-948b-daf1bebab5fc","Type":"ContainerStarted","Data":"495fb3af935e510c14097e40b9cec64af0a5c09f536a50c6474c12850b70d524"} Jul 24 13:09:01.237091 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.236997 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" event={"ID":"46b5d990-b4de-47e1-9607-4dcc723020ee","Type":"ContainerStarted","Data":"ba7f9d1d325508206626d1b5fd9c93a88fce9458fe64f007e6deffaa7c687fb9"} Jul 24 13:09:01.251212 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.251117 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4nd5b" event={"ID":"05df2885-2fde-48f1-aaf8-1a820a0d2f2a","Type":"ContainerStarted","Data":"3f7977db1f4dd780de6962b4c12ab8acd70640132049315b82f57e325c467397"} Jul 24 13:09:01.253555 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.253503 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerStarted","Data":"f3c520b9a5858b23f5d35b42e37730da7c05a1ff57f8b0a1ff07049796a4251a"} Jul 24 13:09:01.260653 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.260567 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k9vv9" event={"ID":"ebd3b024-b11e-4db8-94c9-4344ba779060","Type":"ContainerStarted","Data":"23a2e90c667581cd3548e29eaba52e9d620d3fbdac5705cf93136c1caa117159"} Jul 24 13:09:01.269765 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.269732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" event={"ID":"8ceaada0-e85b-46d4-b0a4-7b3088dbc031","Type":"ContainerStarted","Data":"b6d05683ffadc4b4fb2adddde3c5d152ee4a9fc4d4598978d4a8922dd797acda"} Jul 24 13:09:01.288749 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.288700 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"e84227ed8c29e563e7d20005c9a171455c0927e9aa1fae86b567441f29530f05"} Jul 24 13:09:01.296809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.296763 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87xcs" event={"ID":"595f165a-81a3-4662-9fb1-6e5e7c7dacb9","Type":"ContainerStarted","Data":"b7a1c3b124144fa72789d6b90f3a2ad394e64039972e3e9d7644709c0909c88f"} Jul 24 13:09:01.309014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:01.308844 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bz7bk" event={"ID":"35bcce45-7c4a-4313-b746-8d124094ebc6","Type":"ContainerStarted","Data":"93d50954aa5fb7b899b195a74eae87a0c1fabdad0155c9dd16bca396dc22f134"} Jul 24 13:09:02.190234 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:02.190126 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:02.190785 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.190297 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:02.340553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:02.340428 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" event={"ID":"71e034ab1388ffeb81ee87fb6369df72","Type":"ContainerStarted","Data":"eed223a541549fc355a29680a2a0859d89ff4e40dba0b2d0e62d929de07aa006"} Jul 24 13:09:02.648006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:02.647911 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:02.648192 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.648091 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:02.648192 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.648161 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:09:06.648139458 +0000 UTC m=+10.012063536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:02.748707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:02.748823 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.748950 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.748965 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.748988 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.749004 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.749031 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:09:06.749010115 +0000 UTC m=+10.112934205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:02.749094 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:02.749061 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:06.749043721 +0000 UTC m=+10.112967795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:03.190729 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:03.190176 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:03.190729 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:03.190331 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:03.191571 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:03.191365 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:03.191571 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:03.191519 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:04.190330 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:04.190299 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:04.190555 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:04.190401 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:05.191752 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:05.191199 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:05.191752 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:05.191350 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:05.191752 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:05.191667 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:05.192343 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:05.191800 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:06.190631 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:06.190535 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:06.190803 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.190696 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:06.687782 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:06.687617 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:06.688355 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.687834 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:06.688355 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.687908 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:09:14.687885271 +0000 UTC m=+18.051809343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:06.788756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:06.788673 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:06.788964 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.788908 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:06.788964 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.788939 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:06.788964 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.788954 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:06.789138 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.789020 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:14.789000413 +0000 UTC m=+18.152924491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:06.789646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:06.789386 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:06.789646 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.789559 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:06.789646 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:06.789613 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:09:14.78959627 +0000 UTC m=+18.153520357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:07.191509 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:07.191450 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:07.191693 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:07.191596 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:07.191693 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:07.191626 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:07.191833 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:07.191754 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:08.190643 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:08.190122 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:08.190643 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:08.190269 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:09.190306 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:09.190201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:09.190306 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:09.190201 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:09.190537 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:09.190376 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:09.190537 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:09.190465 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:10.190363 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:10.189967 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:10.190363 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:10.190107 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:11.190832 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:11.190793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:11.191250 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:11.190793 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:11.191250 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:11.190946 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:11.191250 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:11.191007 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:12.189953 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:12.189916 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:12.190124 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:12.190033 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:13.190146 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:13.190106 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:13.190146 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:13.190130 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:13.190678 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:13.190252 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:13.190678 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:13.190413 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:14.190522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:14.190486 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:14.191031 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.190620 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:14.755172 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:14.755134 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:14.755356 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.755333 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:14.755446 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.755433 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:09:30.755409944 +0000 UTC m=+34.119334025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:14.856419 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:14.856364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:14.856600 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:14.856432 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:14.856600 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.856546 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:14.856600 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.856559 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:14.856600 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.856583 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:14.856600 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.856593 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:14.856771 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.856618 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:09:30.856597852 +0000 UTC m=+34.220521927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:14.856771 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:14.856636 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:30.856628767 +0000 UTC m=+34.220552842 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:15.189910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:15.189874 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:15.190095 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:15.190021 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:15.190095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:15.190067 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:15.190181 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:15.190164 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:16.190719 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:16.190676 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:16.191178 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:16.190827 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:17.190833 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:17.190708 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:17.191351 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:17.190841 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:17.191351 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:17.190907 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:17.191351 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:17.191034 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:18.189956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:18.189918 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:18.190129 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:18.190041 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:19.189998 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:19.189965 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:19.190475 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:19.189964 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:19.190475 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:19.190084 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:19.190475 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:19.190193 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:20.190489 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.190150 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:20.191249 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:20.190581 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:20.377943 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.377905 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zm4hb" event={"ID":"077ed00b-9339-4528-948b-daf1bebab5fc","Type":"ContainerStarted","Data":"4f02d6863439658accbe29b62c32c9e25f4efac30b28af8471b26d5c20e6f268"} Jul 24 13:09:20.379591 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.379529 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" event={"ID":"46b5d990-b4de-47e1-9607-4dcc723020ee","Type":"ContainerStarted","Data":"8e822beb8e1591db5e6cac68937d7b9e15c44ae74a7a1cb1e331c0739492cb99"} Jul 24 13:09:20.381118 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.381086 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4nd5b" event={"ID":"05df2885-2fde-48f1-aaf8-1a820a0d2f2a","Type":"ContainerStarted","Data":"966da24fd34ecb01ec3f9a69995aebd533dd43b0e9c1f0881c8c14fa39978a3d"} Jul 24 13:09:20.382674 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.382649 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15b19fb-dfbd-44af-8773-abe59fece774" containerID="08966ad16c12352f83b6a2f7a2419c38c4d81dc9d51e51a2d729af535c082d3d" exitCode=0 Jul 24 13:09:20.382819 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.382718 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerDied","Data":"08966ad16c12352f83b6a2f7a2419c38c4d81dc9d51e51a2d729af535c082d3d"} Jul 24 13:09:20.384275 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.384249 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k9vv9" event={"ID":"ebd3b024-b11e-4db8-94c9-4344ba779060","Type":"ContainerStarted","Data":"4470ea985118d30aaef2ebb7239f0ca6e52b99c50d33e27ba8c58e7590ea064b"} Jul 24 13:09:20.385918 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.385892 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" event={"ID":"8ceaada0-e85b-46d4-b0a4-7b3088dbc031","Type":"ContainerStarted","Data":"b5cf80507eeb675795253933d7e5bd15490fadd3a1f66f81e64ddefcf94ffe9e"} Jul 24 13:09:20.389285 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.389253 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"62703a5cde4514c3dd872fbdd766ccfb798b9ccec58d877d86efe5d83b0401a0"} Jul 24 13:09:20.389429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.389324 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"b46f2643325dc7be765d6e09882f2e94faed107427209ef3eed675329a5f2846"} Jul 24 13:09:20.389429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.389375 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"50792bb9924dc587494e0bb1665eeb73e188251516f3634403daa4f81e9b8601"} Jul 24 13:09:20.389524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.389430 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"61df6a3bf24a317f99a51970aeb6317e7d0779f0b448de90bc95b6923b18ab87"} Jul 24 13:09:20.389524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.389468 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"d88e2efc6b73cc72899995e073f221914171d27caa50919f4079240ef237ce06"} Jul 24 13:09:20.389524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.389504 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"7d948a1ffea39e8ddd9cad8fd10765af2eff9ddc42127d97d9986c32b41a2252"} Jul 24 13:09:20.390820 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.390796 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87xcs" event={"ID":"595f165a-81a3-4662-9fb1-6e5e7c7dacb9","Type":"ContainerStarted","Data":"a1a01b0ef5d60fdf7aa8efec325c483a18d0756ed5211448ff395563fc1fcbd0"} Jul 24 13:09:20.393974 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.393920 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zm4hb" podStartSLOduration=7.512241911 podStartE2EDuration="22.393879031s" totalImagesPullingTime="14.88163712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:20.393861376 +0000 UTC m=+23.757785467" watchObservedRunningTime="2026-07-24 13:09:20.393879031 +0000 UTC m=+23.757803124" Jul 24 13:09:20.394127 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.394105 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-210.ec2.internal" podStartSLOduration=22.394096103 podStartE2EDuration="22.394096103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:02.357035893 +0000 UTC m=+5.720959991" watchObservedRunningTime="2026-07-24 13:09:20.394096103 +0000 UTC m=+23.758020193" Jul 24 13:09:20.409902 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.409852 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jw8n5" podStartSLOduration=3.238657149 podStartE2EDuration="22.409835628s" totalImagesPullingTime="19.171178479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:20.409548154 +0000 UTC m=+23.773472245" watchObservedRunningTime="2026-07-24 13:09:20.409835628 +0000 UTC m=+23.773759719" Jul 24 13:09:20.425395 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.425329 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k9vv9" podStartSLOduration=3.232430234 podStartE2EDuration="22.425313973s" totalImagesPullingTime="19.192883739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:20.424868032 +0000 UTC m=+23.788792122" watchObservedRunningTime="2026-07-24 13:09:20.425313973 +0000 UTC m=+23.789238064" Jul 24 13:09:20.442089 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.442033 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4nd5b" podStartSLOduration=3.275927817 podStartE2EDuration="22.44201781s" totalImagesPullingTime="19.166089993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:20.441447659 +0000 UTC m=+23.805371749" watchObservedRunningTime="2026-07-24 13:09:20.44201781 +0000 UTC m=+23.805941900" Jul 24 13:09:20.463213 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:20.463157 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-87xcs" podStartSLOduration=3.2901801600000002 podStartE2EDuration="22.463137997s" totalImagesPullingTime="19.172957837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:20.463069169 +0000 UTC m=+23.826993260" watchObservedRunningTime="2026-07-24 13:09:20.463137997 +0000 UTC m=+23.827062088" Jul 24 13:09:21.190698 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:21.190610 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:21.190698 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:21.190667 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:21.191266 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:21.190757 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:21.191266 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:21.190913 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:21.394677 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:21.394630 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bz7bk" event={"ID":"35bcce45-7c4a-4313-b746-8d124094ebc6","Type":"ContainerStarted","Data":"30e1231b024267719519806e64d4be1bf2167f3ec213fddc0d35b2161c602f12"} Jul 24 13:09:21.410328 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:21.410258 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bz7bk" podStartSLOduration=8.519268014 podStartE2EDuration="23.410236011s" totalImagesPullingTime="14.890967997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:21.409878689 +0000 UTC m=+24.773802779" watchObservedRunningTime="2026-07-24 13:09:21.410236011 +0000 UTC m=+24.774160105" Jul 24 13:09:21.612189 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:21.612046 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 13:09:22.068219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.068047 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T13:09:21.612063633Z","UUID":"82bc1e69-5fe9-4df0-9a33-75af73fc46cc","Handler":null,"Name":"","Endpoint":""} Jul 24 13:09:22.069864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.069839 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 13:09:22.070001 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.069907 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 13:09:22.162777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.162737 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:09:22.163424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.163398 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:09:22.190593 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.190550 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:22.190788 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:22.190691 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:22.400474 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.400443 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" event={"ID":"8ceaada0-e85b-46d4-b0a4-7b3088dbc031","Type":"ContainerStarted","Data":"ce44f49ed90dbe38645a16dbf0b89600f8399cdd8c5517f7519fc63709cf7832"} Jul 24 13:09:22.400663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.400504 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:09:22.401090 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:22.401068 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4nd5b" Jul 24 13:09:23.190677 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:23.190342 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:23.190832 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:23.190408 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:23.190832 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:23.190746 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:23.191185 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:23.190833 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:23.404771 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:23.404732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" event={"ID":"8ceaada0-e85b-46d4-b0a4-7b3088dbc031","Type":"ContainerStarted","Data":"418571bd8bdfa8a3472515869220e3eccc06f8aeb2b9d9d2a417378c667fddbb"} Jul 24 13:09:23.408547 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:23.408514 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"e4f9cae537e991f6e9e7852693558f8d560703d4940500d91d6e9a406e9cb2a4"} Jul 24 13:09:23.423757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:23.423687 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-56bhw" podStartSLOduration=3.208388034 podStartE2EDuration="25.423663992s" totalImagesPullingTime="22.215275958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:23.42310993 +0000 UTC m=+26.787034021" watchObservedRunningTime="2026-07-24 13:09:23.423663992 +0000 UTC m=+26.787588085" Jul 24 13:09:24.190355 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:24.190311 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:24.190577 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:24.190481 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:25.190684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.190650 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:25.190684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.190683 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:25.191539 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:25.190808 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:25.191539 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:25.190937 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:25.417198 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.416965 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerStarted","Data":"d45eb88ab5add0eb030380f84366fa17276fefc2bdcb4607e45b7f1bf346341a"} Jul 24 13:09:25.421171 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.421140 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" event={"ID":"24987975-7733-4719-aab2-ca011b0f4e86","Type":"ContainerStarted","Data":"a9971332d5358ca656657aa981938bcbf367abf96c4ec3ae091a788a148a1d93"} Jul 24 13:09:25.421494 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.421470 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:09:25.421664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.421548 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:09:25.421664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.421601 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:09:25.438495 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.438399 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:09:25.438638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.438616 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:09:25.473089 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:25.473033 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" podStartSLOduration=8.152307849 podStartE2EDuration="27.473016793s" totalImagesPullingTime="19.320708944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:25.47262209 +0000 UTC m=+28.836546180" watchObservedRunningTime="2026-07-24 13:09:25.473016793 +0000 UTC m=+28.836940883" Jul 24 13:09:26.189746 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:26.189707 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:26.189905 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:26.189812 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:26.425353 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:26.425311 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15b19fb-dfbd-44af-8773-abe59fece774" containerID="d45eb88ab5add0eb030380f84366fa17276fefc2bdcb4607e45b7f1bf346341a" exitCode=0 Jul 24 13:09:26.425969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:26.425416 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerDied","Data":"d45eb88ab5add0eb030380f84366fa17276fefc2bdcb4607e45b7f1bf346341a"} Jul 24 13:09:27.191488 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.191452 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:27.191488 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.191488 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:27.191726 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:27.191559 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:27.191726 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:27.191685 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:27.480596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.480568 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8jsc6"] Jul 24 13:09:27.480949 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.480689 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:27.480949 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:27.480812 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:27.484450 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.484418 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pnd68"] Jul 24 13:09:27.484578 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.484538 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:27.484674 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:27.484649 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:27.484815 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.484793 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdkhm"] Jul 24 13:09:27.484891 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:27.484870 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:27.484891 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:27.484961 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:28.431981 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:28.431943 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15b19fb-dfbd-44af-8773-abe59fece774" containerID="f2b420aabb7f556d2feb64618958511a1420a500cb7527c02cdbcaabd06b8661" exitCode=0 Jul 24 13:09:28.431981 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:28.431986 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerDied","Data":"f2b420aabb7f556d2feb64618958511a1420a500cb7527c02cdbcaabd06b8661"} Jul 24 13:09:29.190252 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:29.189976 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:29.190693 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:29.190031 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:29.190693 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:29.190363 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:29.190693 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:29.190054 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:29.190693 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:29.190456 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:29.190693 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:29.190511 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:30.440051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:30.440015 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15b19fb-dfbd-44af-8773-abe59fece774" containerID="fa144c40c2114f29a6512802205e978ca6e55aee0217a9c0c29752dd73a5f4b6" exitCode=0 Jul 24 13:09:30.440644 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:30.440078 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerDied","Data":"fa144c40c2114f29a6512802205e978ca6e55aee0217a9c0c29752dd73a5f4b6"} Jul 24 13:09:30.778687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:30.778588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:30.778848 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.778745 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:30.778848 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.778809 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs podName:bdb9d81c-3072-4b3b-a7f8-e6bf692e622e nodeName:}" failed. No retries permitted until 2026-07-24 13:10:02.778791872 +0000 UTC m=+66.142715951 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs") pod "network-metrics-daemon-pnd68" (UID: "bdb9d81c-3072-4b3b-a7f8-e6bf692e622e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:30.879619 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:30.879576 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:30.879817 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:30.879631 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:30.879817 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.879739 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:30.879817 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.879747 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:30.879817 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.879769 2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:30.879817 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.879782 2558 projected.go:196] Error preparing data for projected volume kube-api-access-7fs4g for pod openshift-network-diagnostics/network-check-target-sdkhm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:30.879817 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.879810 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret podName:4971787d-2c92-40a4-94d9-6ab28883e02b nodeName:}" failed. No retries permitted until 2026-07-24 13:10:02.879793099 +0000 UTC m=+66.243717168 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret") pod "global-pull-secret-syncer-8jsc6" (UID: "4971787d-2c92-40a4-94d9-6ab28883e02b") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:30.880108 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:30.879833 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g podName:46e66420-02a4-4eb9-83ba-869350f09cc1 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:02.879822333 +0000 UTC m=+66.243746405 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7fs4g" (UniqueName: "kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g") pod "network-check-target-sdkhm" (UID: "46e66420-02a4-4eb9-83ba-869350f09cc1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:31.189928 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:31.189894 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:31.190111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:31.189894 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:31.190111 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:31.190044 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:31.190111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:31.189894 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:31.190267 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:31.190121 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:31.190267 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:31.190162 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:33.190344 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.190298 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:33.191095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.190298 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:33.191095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.190299 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:33.191095 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.190504 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdkhm" podUID="46e66420-02a4-4eb9-83ba-869350f09cc1" Jul 24 13:09:33.191095 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.190561 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pnd68" podUID="bdb9d81c-3072-4b3b-a7f8-e6bf692e622e" Jul 24 13:09:33.191095 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.190462 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jsc6" podUID="4971787d-2c92-40a4-94d9-6ab28883e02b" Jul 24 13:09:33.499547 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.499518 2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-210.ec2.internal" event="NodeReady" Jul 24 13:09:33.499745 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.499664 2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 13:09:33.535736 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.535582 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc"] Jul 24 13:09:33.538126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.538094 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-69k7c"] Jul 24 13:09:33.538294 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.538236 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:33.540436 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.540407 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86db5847b7-cbr7q"] Jul 24 13:09:33.540607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.540495 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.541329 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.541305 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 24 13:09:33.542474 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.542453 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk"] Jul 24 13:09:33.542617 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.542588 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.543221 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.543170 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.543405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.543367 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.544691 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.544673 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4"] Jul 24 13:09:33.544814 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.544797 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.546606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.546587 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6b84687b8c-kmckn"] Jul 24 13:09:33.546836 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.546819 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.548507 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.548485 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57bf4c6dfb-zscvm"] Jul 24 13:09:33.548706 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.548662 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.550537 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.550399 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x4rk7\"" Jul 24 13:09:33.550639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.550592 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 24 13:09:33.550708 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.550641 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 24 13:09:33.550708 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.550645 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-4sl4t"] Jul 24 13:09:33.550809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.550717 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.551528 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.551502 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.551642 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.551527 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 13:09:33.555506 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.553493 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 13:09:33.555506 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.553853 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.557539 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.557516 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct"] Jul 24 13:09:33.557650 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.556543 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-74vbr\"" Jul 24 13:09:33.557710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.557655 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cfzrs\"" Jul 24 13:09:33.557848 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.557812 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kvvpn\"" Jul 24 13:09:33.557848 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.557836 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 24 13:09:33.557961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.557927 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.557961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.557952 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558066 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558116 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558137 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558071 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558254 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558296 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558344 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558355 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 24 13:09:33.558467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558375 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.558967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558546 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.558967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558579 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 24 13:09:33.558967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558586 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.558967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558822 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.558967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558897 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-cswzm\"" Jul 24 13:09:33.559205 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.558990 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sz6sn\"" Jul 24 13:09:33.559445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.559427 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6"] Jul 24 13:09:33.559526 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.559494 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" Jul 24 13:09:33.562929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.561131 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2"] Jul 24 13:09:33.562929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.562252 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:33.562929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.562791 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 13:09:33.563976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.563957 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 24 13:09:33.564692 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.564672 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh"] Jul 24 13:09:33.565090 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.565073 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" Jul 24 13:09:33.567601 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.567364 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86db5847b7-cbr7q"] Jul 24 13:09:33.567601 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.567491 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.569367 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-69k7c"] Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.569712 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.569933 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.570073 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-spl9l\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.570280 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.571234 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmgtg\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.571567 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7v2ls\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.571819 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.572014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.572026 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 24 13:09:33.572669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.572273 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.572669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.572502 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 24 13:09:33.572669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.569967 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.573376 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.573201 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.573559 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.573536 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 24 13:09:33.573850 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.573829 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qzxz5\"" Jul 24 13:09:33.574320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.573963 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 24 13:09:33.574320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.573899 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.574320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.574256 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.574541 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.574364 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pbssp\"" Jul 24 13:09:33.574541 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.574454 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 24 13:09:33.575064 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.575043 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc"] Jul 24 13:09:33.576452 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.576373 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4"] Jul 24 13:09:33.577170 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.576964 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct"] Jul 24 13:09:33.580694 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.580666 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk"] Jul 24 13:09:33.581010 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.580988 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 24 13:09:33.581185 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.581043 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6"] Jul 24 13:09:33.584880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.584857 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2"] Jul 24 13:09:33.585902 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.585878 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nwrp8"] Jul 24 13:09:33.591463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.591124 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b84687b8c-kmckn"] Jul 24 13:09:33.591463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.591267 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.591804 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.591786 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57bf4c6dfb-zscvm"] Jul 24 13:09:33.592759 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.592737 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z9vkb"] Jul 24 13:09:33.595201 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.595175 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.595910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.595727 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh"] Jul 24 13:09:33.596053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.596037 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-4sl4t"] Jul 24 13:09:33.596541 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.596519 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwrp8"] Jul 24 13:09:33.596710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.596614 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 13:09:33.597190 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.597163 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t8pk6\"" Jul 24 13:09:33.597359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.597248 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 13:09:33.597439 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.597375 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 13:09:33.598033 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.597783 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 13:09:33.598288 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.598265 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 13:09:33.598378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.598310 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n54jn\"" Jul 24 13:09:33.610249 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.610210 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z9vkb"] Jul 24 13:09:33.701895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.701760 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-certificates\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.701895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.701825 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-bound-sa-token\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.701895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.701859 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-default-certificate\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.701895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.701890 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0825d55b-6343-411d-8a8c-9dcc1d46bf93-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.701920 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-image-registry-private-configuration\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.701947 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-installation-pull-secrets\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702045 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702098 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/253c7174-7697-4369-85c6-b0f3abe58c64-ca-trust-extracted\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702133 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b2036d-7e89-44e0-9b58-dd8a30691fd8-tmp\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702158 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b2036d-7e89-44e0-9b58-dd8a30691fd8-service-ca-bundle\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702186 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13956c89-0f25-43f9-be04-ab203a8de1f9-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-installation-pull-secrets\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.702269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702247 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702272 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702299 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5db3487-f9e9-412c-a5b3-d4e661de0280-tmp-dir\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702328 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kr6w\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-kube-api-access-5kr6w\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702411 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c393ace-cf46-4319-8dc3-548b7bf80ce0-cert\") pod \"ingress-canary-nwrp8\" (UID: \"2c393ace-cf46-4319-8dc3-548b7bf80ce0\") " pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702468 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/279aea8c-817d-4897-b402-9da959d25728-trusted-ca\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b2036d-7e89-44e0-9b58-dd8a30691fd8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702527 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-image-registry-private-configuration\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702598 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-registry-certificates\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702630 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78h2f\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-kube-api-access-78h2f\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702660 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31c6e268-c2a1-49b3-b910-e185014fcef8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702684 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-stats-auth\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.702709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/46b2036d-7e89-44e0-9b58-dd8a30691fd8-snapshots\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702749 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46b2036d-7e89-44e0-9b58-dd8a30691fd8-serving-cert\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702812 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4jp\" (UniqueName: \"kubernetes.io/projected/552b5999-624e-4856-b799-542d612da9cd-kube-api-access-8z4jp\") pod \"network-check-source-7d88cb55d8-59zm2\" (UID: \"552b5999-624e-4856-b799-542d612da9cd\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702845 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c4d641-690d-4043-830f-9b2dfd0dabbb-ca-trust-extracted\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702880 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.702949 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9b3afa1c-c462-4ce1-891c-7412da5b4566-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703021 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw6km\" (UniqueName: \"kubernetes.io/projected/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-kube-api-access-qw6km\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703050 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4frm\" (UniqueName: \"kubernetes.io/projected/279aea8c-817d-4897-b402-9da959d25728-kube-api-access-m4frm\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703077 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2hl\" (UniqueName: \"kubernetes.io/projected/b5db3487-f9e9-412c-a5b3-d4e661de0280-kube-api-access-5s2hl\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703105 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/279aea8c-817d-4897-b402-9da959d25728-config\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703127 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b5db3487-f9e9-412c-a5b3-d4e661de0280-config-volume\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.703219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j977v\" (UniqueName: \"kubernetes.io/projected/13956c89-0f25-43f9-be04-ab203a8de1f9-kube-api-access-j977v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703260 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ktp\" (UniqueName: \"kubernetes.io/projected/2c393ace-cf46-4319-8dc3-548b7bf80ce0-kube-api-access-w9ktp\") pod \"ingress-canary-nwrp8\" (UID: \"2c393ace-cf46-4319-8dc3-548b7bf80ce0\") " pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8wvs\" (UniqueName: \"kubernetes.io/projected/a7f709b4-e9e2-428f-9c9c-ef397ad87868-kube-api-access-f8wvs\") pod \"volume-data-source-validator-549c6d54cf-sr5ct\" (UID: \"a7f709b4-e9e2-428f-9c9c-ef397ad87868\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703336 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703361 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwlvh\" (UniqueName: \"kubernetes.io/projected/46b2036d-7e89-44e0-9b58-dd8a30691fd8-kube-api-access-cwlvh\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703377 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-bound-sa-token\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703415 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703441 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b6t7\" (UniqueName: \"kubernetes.io/projected/9b3afa1c-c462-4ce1-891c-7412da5b4566-kube-api-access-8b6t7\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703464 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-trusted-ca\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703481 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkgc9\" (UniqueName: \"kubernetes.io/projected/31c6e268-c2a1-49b3-b910-e185014fcef8-kube-api-access-pkgc9\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703524 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-trusted-ca\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703576 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31c6e268-c2a1-49b3-b910-e185014fcef8-config\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703608 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5p4\" (UniqueName: \"kubernetes.io/projected/3d842ba9-bc55-4f50-a760-399f427bb42d-kube-api-access-xt5p4\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:33.703646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703637 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.704114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703666 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/279aea8c-817d-4897-b402-9da959d25728-serving-cert\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.704114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.703695 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13956c89-0f25-43f9-be04-ab203a8de1f9-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.804295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804209 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31c6e268-c2a1-49b3-b910-e185014fcef8-config\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.804295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804250 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5p4\" (UniqueName: \"kubernetes.io/projected/3d842ba9-bc55-4f50-a760-399f427bb42d-kube-api-access-xt5p4\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:33.804295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.804295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804289 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/279aea8c-817d-4897-b402-9da959d25728-serving-cert\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804310 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13956c89-0f25-43f9-be04-ab203a8de1f9-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804338 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-certificates\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804365 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-bound-sa-token\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804411 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-default-certificate\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.804428 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.804550 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.304527273 +0000 UTC m=+37.668451360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : secret "router-metrics-certs-default" not found Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804436 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0825d55b-6343-411d-8a8c-9dcc1d46bf93-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804602 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-image-registry-private-configuration\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804637 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-installation-pull-secrets\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804669 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:33.804711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804696 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/253c7174-7697-4369-85c6-b0f3abe58c64-ca-trust-extracted\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804728 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b2036d-7e89-44e0-9b58-dd8a30691fd8-tmp\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804755 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b2036d-7e89-44e0-9b58-dd8a30691fd8-service-ca-bundle\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804781 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13956c89-0f25-43f9-be04-ab203a8de1f9-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804812 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-installation-pull-secrets\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804842 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804872 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804897 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5db3487-f9e9-412c-a5b3-d4e661de0280-tmp-dir\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804928 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kr6w\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-kube-api-access-5kr6w\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804952 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c393ace-cf46-4319-8dc3-548b7bf80ce0-cert\") pod \"ingress-canary-nwrp8\" (UID: \"2c393ace-cf46-4319-8dc3-548b7bf80ce0\") " pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804979 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/279aea8c-817d-4897-b402-9da959d25728-trusted-ca\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.804982 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-certificates\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805006 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b2036d-7e89-44e0-9b58-dd8a30691fd8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805010 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31c6e268-c2a1-49b3-b910-e185014fcef8-config\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805010 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13956c89-0f25-43f9-be04-ab203a8de1f9-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805034 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.805264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805081 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-image-registry-private-configuration\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805112 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-registry-certificates\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805141 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78h2f\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-kube-api-access-78h2f\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805172 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31c6e268-c2a1-49b3-b910-e185014fcef8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805200 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-stats-auth\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805225 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/46b2036d-7e89-44e0-9b58-dd8a30691fd8-snapshots\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805249 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46b2036d-7e89-44e0-9b58-dd8a30691fd8-serving-cert\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805277 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4jp\" (UniqueName: \"kubernetes.io/projected/552b5999-624e-4856-b799-542d612da9cd-kube-api-access-8z4jp\") pod \"network-check-source-7d88cb55d8-59zm2\" (UID: \"552b5999-624e-4856-b799-542d612da9cd\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805319 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c4d641-690d-4043-830f-9b2dfd0dabbb-ca-trust-extracted\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805329 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/253c7174-7697-4369-85c6-b0f3abe58c64-ca-trust-extracted\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805354 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805405 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9b3afa1c-c462-4ce1-891c-7412da5b4566-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805441 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw6km\" (UniqueName: \"kubernetes.io/projected/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-kube-api-access-qw6km\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805476 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4frm\" (UniqueName: \"kubernetes.io/projected/279aea8c-817d-4897-b402-9da959d25728-kube-api-access-m4frm\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805505 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2hl\" (UniqueName: \"kubernetes.io/projected/b5db3487-f9e9-412c-a5b3-d4e661de0280-kube-api-access-5s2hl\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805535 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0825d55b-6343-411d-8a8c-9dcc1d46bf93-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:33.805983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805538 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/279aea8c-817d-4897-b402-9da959d25728-config\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805582 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b5db3487-f9e9-412c-a5b3-d4e661de0280-config-volume\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805614 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j977v\" (UniqueName: \"kubernetes.io/projected/13956c89-0f25-43f9-be04-ab203a8de1f9-kube-api-access-j977v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805693 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ktp\" (UniqueName: \"kubernetes.io/projected/2c393ace-cf46-4319-8dc3-548b7bf80ce0-kube-api-access-w9ktp\") pod \"ingress-canary-nwrp8\" (UID: \"2c393ace-cf46-4319-8dc3-548b7bf80ce0\") " pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805720 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8wvs\" (UniqueName: \"kubernetes.io/projected/a7f709b4-e9e2-428f-9c9c-ef397ad87868-kube-api-access-f8wvs\") pod \"volume-data-source-validator-549c6d54cf-sr5ct\" (UID: \"a7f709b4-e9e2-428f-9c9c-ef397ad87868\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805758 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805780 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwlvh\" (UniqueName: \"kubernetes.io/projected/46b2036d-7e89-44e0-9b58-dd8a30691fd8-kube-api-access-cwlvh\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805804 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-bound-sa-token\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805831 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805853 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b6t7\" (UniqueName: \"kubernetes.io/projected/9b3afa1c-c462-4ce1-891c-7412da5b4566-kube-api-access-8b6t7\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805878 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-trusted-ca\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805899 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkgc9\" (UniqueName: \"kubernetes.io/projected/31c6e268-c2a1-49b3-b910-e185014fcef8-kube-api-access-pkgc9\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.806684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.805938 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-trusted-ca\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.807224 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.807165 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/279aea8c-817d-4897-b402-9da959d25728-config\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.807469 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.807444 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/46b2036d-7e89-44e0-9b58-dd8a30691fd8-tmp\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.807979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.807951 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b2036d-7e89-44e0-9b58-dd8a30691fd8-service-ca-bundle\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.808366 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-trusted-ca\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.808411 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.808504 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls podName:3d842ba9-bc55-4f50-a760-399f427bb42d nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.308474302 +0000 UTC m=+37.672398373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls") pod "cluster-samples-operator-6748489656-rsmhc" (UID: "3d842ba9-bc55-4f50-a760-399f427bb42d") : secret "samples-operator-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.809348 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9b3afa1c-c462-4ce1-891c-7412da5b4566-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.809434 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b5db3487-f9e9-412c-a5b3-d4e661de0280-config-volume\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.809452 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.809529 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls podName:b5db3487-f9e9-412c-a5b3-d4e661de0280 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.309508846 +0000 UTC m=+37.673432915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls") pod "dns-default-z9vkb" (UID: "b5db3487-f9e9-412c-a5b3-d4e661de0280") : secret "dns-default-metrics-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.809839 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c4d641-690d-4043-830f-9b2dfd0dabbb-ca-trust-extracted\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.809989 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.309963903 +0000 UTC m=+37.673887993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : configmap references non-existent config key: service-ca.crt Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810198 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810451 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert podName:0825d55b-6343-411d-8a8c-9dcc1d46bf93 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.310428139 +0000 UTC m=+37.674352260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-5vfs6" (UID: "0825d55b-6343-411d-8a8c-9dcc1d46bf93") : secret "networking-console-plugin-cert" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810533 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810551 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57bf4c6dfb-zscvm: secret "image-registry-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810673 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:33.811045 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810735 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls podName:9b3afa1c-c462-4ce1-891c-7412da5b4566 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.31071266 +0000 UTC m=+37.674636751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sbzh4" (UID: "9b3afa1c-c462-4ce1-891c-7412da5b4566") : secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:33.811934 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.810795 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls podName:253c7174-7697-4369-85c6-b0f3abe58c64 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.310768297 +0000 UTC m=+37.674692368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls") pod "image-registry-57bf4c6dfb-zscvm" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64") : secret "image-registry-tls" not found Jul 24 13:09:33.811934 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.811076 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5db3487-f9e9-412c-a5b3-d4e661de0280-tmp-dir\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.811934 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.811372 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-installation-pull-secrets\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.811934 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.811926 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-registry-certificates\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.812436 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/46b2036d-7e89-44e0-9b58-dd8a30691fd8-snapshots\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.814054 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-trusted-ca\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.814177 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.814195 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86db5847b7-cbr7q: secret "image-registry-tls" not found Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:33.814271 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls podName:d0c4d641-690d-4043-830f-9b2dfd0dabbb nodeName:}" failed. No retries permitted until 2026-07-24 13:09:34.31424433 +0000 UTC m=+37.678168403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls") pod "image-registry-86db5847b7-cbr7q" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb") : secret "image-registry-tls" not found Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.814648 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/279aea8c-817d-4897-b402-9da959d25728-trusted-ca\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.815020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.814759 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31c6e268-c2a1-49b3-b910-e185014fcef8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.815615 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.815562 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46b2036d-7e89-44e0-9b58-dd8a30691fd8-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.817660 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.817631 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c393ace-cf46-4319-8dc3-548b7bf80ce0-cert\") pod \"ingress-canary-nwrp8\" (UID: \"2c393ace-cf46-4319-8dc3-548b7bf80ce0\") " pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.819501 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.818880 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-stats-auth\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.819501 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.819069 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46b2036d-7e89-44e0-9b58-dd8a30691fd8-serving-cert\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.820016 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.819985 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-bound-sa-token\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.822974 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.822945 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4frm\" (UniqueName: \"kubernetes.io/projected/279aea8c-817d-4897-b402-9da959d25728-kube-api-access-m4frm\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.823096 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.823034 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4jp\" (UniqueName: \"kubernetes.io/projected/552b5999-624e-4856-b799-542d612da9cd-kube-api-access-8z4jp\") pod \"network-check-source-7d88cb55d8-59zm2\" (UID: \"552b5999-624e-4856-b799-542d612da9cd\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" Jul 24 13:09:33.823271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.823236 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw6km\" (UniqueName: \"kubernetes.io/projected/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-kube-api-access-qw6km\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.824308 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.823842 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kr6w\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-kube-api-access-5kr6w\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.825979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.825951 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78h2f\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-kube-api-access-78h2f\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.826521 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.826497 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkgc9\" (UniqueName: \"kubernetes.io/projected/31c6e268-c2a1-49b3-b910-e185014fcef8-kube-api-access-pkgc9\") pod \"service-ca-operator-5d6c75f84f-k8bnk\" (UID: \"31c6e268-c2a1-49b3-b910-e185014fcef8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.826815 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.826788 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b6t7\" (UniqueName: \"kubernetes.io/projected/9b3afa1c-c462-4ce1-891c-7412da5b4566-kube-api-access-8b6t7\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.830238 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5p4\" (UniqueName: \"kubernetes.io/projected/3d842ba9-bc55-4f50-a760-399f427bb42d-kube-api-access-xt5p4\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.830467 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13956c89-0f25-43f9-be04-ab203a8de1f9-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.830638 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/279aea8c-817d-4897-b402-9da959d25728-serving-cert\") pod \"console-operator-584dcb6b54-4sl4t\" (UID: \"279aea8c-817d-4897-b402-9da959d25728\") " pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.830790 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-bound-sa-token\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.831224 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-installation-pull-secrets\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.831226 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j977v\" (UniqueName: \"kubernetes.io/projected/13956c89-0f25-43f9-be04-ab203a8de1f9-kube-api-access-j977v\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh\" (UID: \"13956c89-0f25-43f9-be04-ab203a8de1f9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.831330 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwlvh\" (UniqueName: \"kubernetes.io/projected/46b2036d-7e89-44e0-9b58-dd8a30691fd8-kube-api-access-cwlvh\") pod \"insights-operator-5885fdb759-69k7c\" (UID: \"46b2036d-7e89-44e0-9b58-dd8a30691fd8\") " pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.831675 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.831560 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-image-registry-private-configuration\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:33.832178 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.832042 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-image-registry-private-configuration\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:33.833683 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.833658 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-default-certificate\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:33.834057 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.834000 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2hl\" (UniqueName: \"kubernetes.io/projected/b5db3487-f9e9-412c-a5b3-d4e661de0280-kube-api-access-5s2hl\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:33.835059 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.835032 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8wvs\" (UniqueName: \"kubernetes.io/projected/a7f709b4-e9e2-428f-9c9c-ef397ad87868-kube-api-access-f8wvs\") pod \"volume-data-source-validator-549c6d54cf-sr5ct\" (UID: \"a7f709b4-e9e2-428f-9c9c-ef397ad87868\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" Jul 24 13:09:33.841120 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.841077 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ktp\" (UniqueName: \"kubernetes.io/projected/2c393ace-cf46-4319-8dc3-548b7bf80ce0-kube-api-access-w9ktp\") pod \"ingress-canary-nwrp8\" (UID: \"2c393ace-cf46-4319-8dc3-548b7bf80ce0\") " pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:33.878809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.878522 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-69k7c" Jul 24 13:09:33.886253 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.886205 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" Jul 24 13:09:33.923617 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.923578 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:33.930513 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.930477 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" Jul 24 13:09:33.946403 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.946349 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" Jul 24 13:09:33.953879 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.953849 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" Jul 24 13:09:33.955639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.955607 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:33.963029 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:33.962998 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nwrp8" Jul 24 13:09:34.102843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.102806 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:34.185752 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.185714 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:34.280201 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.280164 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:34.311089 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311050 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:34.311272 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311122 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:34.311272 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311161 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:34.311272 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311193 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:34.311272 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311223 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:34.311272 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311236 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:34.311272 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311262 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57bf4c6dfb-zscvm: secret "image-registry-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311282 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311320 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311397 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311322 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311413 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311259 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311325 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls podName:253c7174-7697-4369-85c6-b0f3abe58c64 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311307398 +0000 UTC m=+38.675231471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls") pod "image-registry-57bf4c6dfb-zscvm" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64") : secret "image-registry-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311489 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311469394 +0000 UTC m=+38.675393473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : secret "router-metrics-certs-default" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311513 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls podName:3d842ba9-bc55-4f50-a760-399f427bb42d nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311502125 +0000 UTC m=+38.675426198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls") pod "cluster-samples-operator-6748489656-rsmhc" (UID: "3d842ba9-bc55-4f50-a760-399f427bb42d") : secret "samples-operator-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311537 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert podName:0825d55b-6343-411d-8a8c-9dcc1d46bf93 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311527868 +0000 UTC m=+38.675451937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-5vfs6" (UID: "0825d55b-6343-411d-8a8c-9dcc1d46bf93") : secret "networking-console-plugin-cert" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311564 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls podName:9b3afa1c-c462-4ce1-891c-7412da5b4566 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311555376 +0000 UTC m=+38.675479445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sbzh4" (UID: "9b3afa1c-c462-4ce1-891c-7412da5b4566") : secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:34.311618 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311606 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls podName:b5db3487-f9e9-412c-a5b3-d4e661de0280 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311581973 +0000 UTC m=+38.675506056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls") pod "dns-default-z9vkb" (UID: "b5db3487-f9e9-412c-a5b3-d4e661de0280") : secret "dns-default-metrics-tls" not found Jul 24 13:09:34.312040 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.311677 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:34.312040 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.311812 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.311797497 +0000 UTC m=+38.675721568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : configmap references non-existent config key: service-ca.crt Jul 24 13:09:34.379830 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.379739 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:34.412921 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.412879 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:34.413114 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.413052 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:34.413114 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.413080 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86db5847b7-cbr7q: secret "image-registry-tls" not found Jul 24 13:09:34.413189 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:34.413138 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls podName:d0c4d641-690d-4043-830f-9b2dfd0dabbb nodeName:}" failed. No retries permitted until 2026-07-24 13:09:35.413122182 +0000 UTC m=+38.777046251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls") pod "image-registry-86db5847b7-cbr7q" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb") : secret "image-registry-tls" not found Jul 24 13:09:34.483105 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.483065 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:34.581023 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.580986 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:34.981871 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.981520 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2"] Jul 24 13:09:34.982773 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.982692 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-69k7c"] Jul 24 13:09:34.987429 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:34.987367 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod552b5999_624e_4856_b799_542d612da9cd.slice/crio-c974fd4eedc623155bc8674c454becf510ef53dc2bc807d909ee20fe918fe7a8 WatchSource:0}: Error finding container c974fd4eedc623155bc8674c454becf510ef53dc2bc807d909ee20fe918fe7a8: Status 404 returned error can't find the container with id c974fd4eedc623155bc8674c454becf510ef53dc2bc807d909ee20fe918fe7a8 Jul 24 13:09:34.990569 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:34.990453 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct"] Jul 24 13:09:34.997176 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:34.997145 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f709b4_e9e2_428f_9c9c_ef397ad87868.slice/crio-361c763ccd31b18d1b1f400db1c7920ad1bc6e569b76fa622766d0fa7eaf55f4 WatchSource:0}: Error finding container 361c763ccd31b18d1b1f400db1c7920ad1bc6e569b76fa622766d0fa7eaf55f4: Status 404 returned error can't find the container with id 361c763ccd31b18d1b1f400db1c7920ad1bc6e569b76fa622766d0fa7eaf55f4 Jul 24 13:09:35.007556 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.007462 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nwrp8"] Jul 24 13:09:35.008692 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.008663 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh"] Jul 24 13:09:35.009716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.009690 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk"] Jul 24 13:09:35.016169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.016138 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-4sl4t"] Jul 24 13:09:35.020461 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:35.020433 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c6e268_c2a1_49b3_b910_e185014fcef8.slice/crio-88d2d62774c6c5b90584db82e76e51cabbab31f44941c7f91a6418e0b92480e0 WatchSource:0}: Error finding container 88d2d62774c6c5b90584db82e76e51cabbab31f44941c7f91a6418e0b92480e0: Status 404 returned error can't find the container with id 88d2d62774c6c5b90584db82e76e51cabbab31f44941c7f91a6418e0b92480e0 Jul 24 13:09:35.022547 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:35.022115 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279aea8c_817d_4897_b402_9da959d25728.slice/crio-2ebfeb3f1ecb199c5983f29025cf00c377b29312b1ff7a05826e981e6bbb6ac8 WatchSource:0}: Error finding container 2ebfeb3f1ecb199c5983f29025cf00c377b29312b1ff7a05826e981e6bbb6ac8: Status 404 returned error can't find the container with id 2ebfeb3f1ecb199c5983f29025cf00c377b29312b1ff7a05826e981e6bbb6ac8 Jul 24 13:09:35.022667 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:35.022613 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13956c89_0f25_43f9_be04_ab203a8de1f9.slice/crio-fac86b0b0f073fb3bab8cdc498723f35afb7d0b13f2d58f7ac6fd0a8b2d03ba5 WatchSource:0}: Error finding container fac86b0b0f073fb3bab8cdc498723f35afb7d0b13f2d58f7ac6fd0a8b2d03ba5: Status 404 returned error can't find the container with id fac86b0b0f073fb3bab8cdc498723f35afb7d0b13f2d58f7ac6fd0a8b2d03ba5 Jul 24 13:09:35.023336 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:35.023307 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c393ace_cf46_4319_8dc3_548b7bf80ce0.slice/crio-6c6dd4975dc5e3ff52587b83e032a2476bc6021976364f0509221455106218e2 WatchSource:0}: Error finding container 6c6dd4975dc5e3ff52587b83e032a2476bc6021976364f0509221455106218e2: Status 404 returned error can't find the container with id 6c6dd4975dc5e3ff52587b83e032a2476bc6021976364f0509221455106218e2 Jul 24 13:09:35.064463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.064424 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:35.165196 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.165161 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:35.189900 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.189862 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:09:35.190052 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.189862 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:09:35.190052 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.189872 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:09:35.192929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.192901 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:09:35.193098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.192991 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rh25q\"" Jul 24 13:09:35.193098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.193011 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dqwnd\"" Jul 24 13:09:35.193098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.193069 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:09:35.333461 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333350 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:35.333461 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333444 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333485 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333543 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333519608 +0000 UTC m=+40.697443689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : configmap references non-existent config key: service-ca.crt Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333578 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333602 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333620 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57bf4c6dfb-zscvm: secret "image-registry-tls" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333621 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333634 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333657 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls podName:253c7174-7697-4369-85c6-b0f3abe58c64 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333645628 +0000 UTC m=+40.697569705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls") pod "image-registry-57bf4c6dfb-zscvm" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64") : secret "image-registry-tls" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333687 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333678558 +0000 UTC m=+40.697602643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : secret "router-metrics-certs-default" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333708 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.333751 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333759 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333783 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333793 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert podName:0825d55b-6343-411d-8a8c-9dcc1d46bf93 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333784293 +0000 UTC m=+40.697708362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-5vfs6" (UID: "0825d55b-6343-411d-8a8c-9dcc1d46bf93") : secret "networking-console-plugin-cert" not found Jul 24 13:09:35.333968 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333825 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls podName:9b3afa1c-c462-4ce1-891c-7412da5b4566 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333812208 +0000 UTC m=+40.697736277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sbzh4" (UID: "9b3afa1c-c462-4ce1-891c-7412da5b4566") : secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:35.334442 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333837 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:09:35.334442 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333847 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:09:35.334442 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333880 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls podName:b5db3487-f9e9-412c-a5b3-d4e661de0280 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333869075 +0000 UTC m=+40.697793147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls") pod "dns-default-z9vkb" (UID: "b5db3487-f9e9-412c-a5b3-d4e661de0280") : secret "dns-default-metrics-tls" not found Jul 24 13:09:35.334442 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.333893 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls podName:3d842ba9-bc55-4f50-a760-399f427bb42d nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.333886483 +0000 UTC m=+40.697810551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls") pod "cluster-samples-operator-6748489656-rsmhc" (UID: "3d842ba9-bc55-4f50-a760-399f427bb42d") : secret "samples-operator-tls" not found Jul 24 13:09:35.434973 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.434939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:35.435122 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.435104 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:35.435122 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.435120 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86db5847b7-cbr7q: secret "image-registry-tls" not found Jul 24 13:09:35.435263 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:35.435178 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls podName:d0c4d641-690d-4043-830f-9b2dfd0dabbb nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.435160742 +0000 UTC m=+40.799084831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls") pod "image-registry-86db5847b7-cbr7q" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb") : secret "image-registry-tls" not found Jul 24 13:09:35.460709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.460665 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15b19fb-dfbd-44af-8773-abe59fece774" containerID="b095b7c98d8217d1840d096a967fdb7763feeb6ffa25adf00a0c8f73c399bf79" exitCode=0 Jul 24 13:09:35.460709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.460699 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerDied","Data":"b095b7c98d8217d1840d096a967fdb7763feeb6ffa25adf00a0c8f73c399bf79"} Jul 24 13:09:35.462145 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.462120 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" event={"ID":"279aea8c-817d-4897-b402-9da959d25728","Type":"ContainerStarted","Data":"2ebfeb3f1ecb199c5983f29025cf00c377b29312b1ff7a05826e981e6bbb6ac8"} Jul 24 13:09:35.463703 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.463675 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" event={"ID":"552b5999-624e-4856-b799-542d612da9cd","Type":"ContainerStarted","Data":"c974fd4eedc623155bc8674c454becf510ef53dc2bc807d909ee20fe918fe7a8"} Jul 24 13:09:35.465245 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.465166 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-69k7c" event={"ID":"46b2036d-7e89-44e0-9b58-dd8a30691fd8","Type":"ContainerStarted","Data":"7a70b14e647120e67b52def6eb0504bb70ec60408a7359c1b9ba97bce4e79c69"} Jul 24 13:09:35.466777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.466749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" event={"ID":"a7f709b4-e9e2-428f-9c9c-ef397ad87868","Type":"ContainerStarted","Data":"361c763ccd31b18d1b1f400db1c7920ad1bc6e569b76fa622766d0fa7eaf55f4"} Jul 24 13:09:35.468089 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.468060 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" event={"ID":"31c6e268-c2a1-49b3-b910-e185014fcef8","Type":"ContainerStarted","Data":"88d2d62774c6c5b90584db82e76e51cabbab31f44941c7f91a6418e0b92480e0"} Jul 24 13:09:35.469298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.469267 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" event={"ID":"13956c89-0f25-43f9-be04-ab203a8de1f9","Type":"ContainerStarted","Data":"fac86b0b0f073fb3bab8cdc498723f35afb7d0b13f2d58f7ac6fd0a8b2d03ba5"} Jul 24 13:09:35.470856 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.470832 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwrp8" event={"ID":"2c393ace-cf46-4319-8dc3-548b7bf80ce0","Type":"ContainerStarted","Data":"6c6dd4975dc5e3ff52587b83e032a2476bc6021976364f0509221455106218e2"} Jul 24 13:09:35.481715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.481615 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:35.570479 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:35.570365 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:36.482508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:36.482053 2558 generic.go:356] "Generic (PLEG): container finished" podID="a15b19fb-dfbd-44af-8773-abe59fece774" containerID="4597b25501de65472cc52f33197791eb49020914cdbab1b3281508fe9e964153" exitCode=0 Jul 24 13:09:36.482508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:36.482377 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerDied","Data":"4597b25501de65472cc52f33197791eb49020914cdbab1b3281508fe9e964153"} Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352447 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352553 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352610 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352656 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352686 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352725 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.352771 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.352936 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.352915577 +0000 UTC m=+44.716839649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : configmap references non-existent config key: service-ca.crt Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353471 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353490 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57bf4c6dfb-zscvm: secret "image-registry-tls" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353545 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls podName:253c7174-7697-4369-85c6-b0f3abe58c64 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.353527309 +0000 UTC m=+44.717451400 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls") pod "image-registry-57bf4c6dfb-zscvm" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64") : secret "image-registry-tls" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353716 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353764 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.353748948 +0000 UTC m=+44.717673017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : secret "router-metrics-certs-default" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353874 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353915 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls podName:3d842ba9-bc55-4f50-a760-399f427bb42d nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.353903402 +0000 UTC m=+44.717827475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls") pod "cluster-samples-operator-6748489656-rsmhc" (UID: "3d842ba9-bc55-4f50-a760-399f427bb42d") : secret "samples-operator-tls" not found Jul 24 13:09:37.354301 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.353981 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 13:09:37.355293 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.354018 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert podName:0825d55b-6343-411d-8a8c-9dcc1d46bf93 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.354007868 +0000 UTC m=+44.717931941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-5vfs6" (UID: "0825d55b-6343-411d-8a8c-9dcc1d46bf93") : secret "networking-console-plugin-cert" not found Jul 24 13:09:37.355293 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.354126 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:37.355293 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.354164 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls podName:9b3afa1c-c462-4ce1-891c-7412da5b4566 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.354152201 +0000 UTC m=+44.718076275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sbzh4" (UID: "9b3afa1c-c462-4ce1-891c-7412da5b4566") : secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:37.355293 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.354234 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:09:37.355293 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.354270 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls podName:b5db3487-f9e9-412c-a5b3-d4e661de0280 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.354259097 +0000 UTC m=+44.718183169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls") pod "dns-default-z9vkb" (UID: "b5db3487-f9e9-412c-a5b3-d4e661de0280") : secret "dns-default-metrics-tls" not found Jul 24 13:09:37.453894 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.453849 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:37.454938 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.454146 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:37.454938 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.454508 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:37.454938 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.454527 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86db5847b7-cbr7q: secret "image-registry-tls" not found Jul 24 13:09:37.454938 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:37.454586 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls podName:d0c4d641-690d-4043-830f-9b2dfd0dabbb nodeName:}" failed. No retries permitted until 2026-07-24 13:09:41.454566957 +0000 UTC m=+44.818491032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls") pod "image-registry-86db5847b7-cbr7q" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb") : secret "image-registry-tls" not found Jul 24 13:09:37.497679 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.497581 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tj56n" event={"ID":"a15b19fb-dfbd-44af-8773-abe59fece774","Type":"ContainerStarted","Data":"d1b065bd115250eac1d22379776b135eea20a083763ddbc82695fb17e487e541"} Jul 24 13:09:37.523376 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.523230 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tj56n" podStartSLOduration=9.027493174 podStartE2EDuration="39.523204474s" totalImagesPullingTime="30.4957113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:37.519233555 +0000 UTC m=+40.883157658" watchObservedRunningTime="2026-07-24 13:09:37.523204474 +0000 UTC m=+40.887128565" Jul 24 13:09:37.656725 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.656631 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:37.717147 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:37.717060 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:39.393803 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:39.393742 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:39.493177 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:39.493143 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:40.632855 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:40.632809 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:40.734555 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:40.734515 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:41.394438 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.394397 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:41.394626 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.394498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:41.394626 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.394551 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:41.394626 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394576 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:41.394626 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.394590 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:41.394626 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394600 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57bf4c6dfb-zscvm: secret "image-registry-tls" not found Jul 24 13:09:41.394883 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394745 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 13:09:41.394883 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394760 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:09:41.394883 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.394825 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394888 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls podName:3d842ba9-bc55-4f50-a760-399f427bb42d nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.394865397 +0000 UTC m=+52.758789473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls") pod "cluster-samples-operator-6748489656-rsmhc" (UID: "3d842ba9-bc55-4f50-a760-399f427bb42d") : secret "samples-operator-tls" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394881 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394923 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394940 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.394929527 +0000 UTC m=+52.758853597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : secret "router-metrics-certs-default" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.394918 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394958 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls podName:253c7174-7697-4369-85c6-b0f3abe58c64 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.394948823 +0000 UTC m=+52.758872892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls") pod "image-registry-57bf4c6dfb-zscvm" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64") : secret "image-registry-tls" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.394980 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls podName:9b3afa1c-c462-4ce1-891c-7412da5b4566 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.394971823 +0000 UTC m=+52.758895892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sbzh4" (UID: "9b3afa1c-c462-4ce1-891c-7412da5b4566") : secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.395004 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:09:41.395028 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.395013 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert podName:0825d55b-6343-411d-8a8c-9dcc1d46bf93 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.394991812 +0000 UTC m=+52.758915904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-5vfs6" (UID: "0825d55b-6343-411d-8a8c-9dcc1d46bf93") : secret "networking-console-plugin-cert" not found Jul 24 13:09:41.395502 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.395038 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls podName:b5db3487-f9e9-412c-a5b3-d4e661de0280 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.395027937 +0000 UTC m=+52.758952005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls") pod "dns-default-z9vkb" (UID: "b5db3487-f9e9-412c-a5b3-d4e661de0280") : secret "dns-default-metrics-tls" not found Jul 24 13:09:41.395502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.395106 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:41.395502 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.395255 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.395239931 +0000 UTC m=+52.759164013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : configmap references non-existent config key: service-ca.crt Jul 24 13:09:41.438654 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.438610 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:41.496202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.496162 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:41.496403 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.496344 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:41.496403 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.496365 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86db5847b7-cbr7q: secret "image-registry-tls" not found Jul 24 13:09:41.496505 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:41.496482 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls podName:d0c4d641-690d-4043-830f-9b2dfd0dabbb nodeName:}" failed. No retries permitted until 2026-07-24 13:09:49.496462891 +0000 UTC m=+52.860386977 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls") pod "image-registry-86db5847b7-cbr7q" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb") : secret "image-registry-tls" not found Jul 24 13:09:41.536503 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.536465 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:41.632743 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.632692 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:41.731969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.731881 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:41.780376 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:41.780339 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:43.517348 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.517313 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/0.log" Jul 24 13:09:43.517920 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.517366 2558 generic.go:356] "Generic (PLEG): container finished" podID="279aea8c-817d-4897-b402-9da959d25728" containerID="d2655f2ed531d42ae2ab9bf3cb83ab0bcbf497c713dfa620cd00cb6be4832803" exitCode=255 Jul 24 13:09:43.517920 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.517478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" event={"ID":"279aea8c-817d-4897-b402-9da959d25728","Type":"ContainerDied","Data":"d2655f2ed531d42ae2ab9bf3cb83ab0bcbf497c713dfa620cd00cb6be4832803"} Jul 24 13:09:43.517920 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.517700 2558 scope.go:117] "RemoveContainer" containerID="d2655f2ed531d42ae2ab9bf3cb83ab0bcbf497c713dfa620cd00cb6be4832803" Jul 24 13:09:43.521884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.521504 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" event={"ID":"552b5999-624e-4856-b799-542d612da9cd","Type":"ContainerStarted","Data":"0d44b474124a6b9faebf27ee46c6c76e4282915ea23471e54681efa544966f18"} Jul 24 13:09:43.523333 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.523292 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-69k7c" event={"ID":"46b2036d-7e89-44e0-9b58-dd8a30691fd8","Type":"ContainerStarted","Data":"b46263db2bbb3f10bfad8656117779d3e124deb15c811376269cae3a0112551f"} Jul 24 13:09:43.525100 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.525076 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" event={"ID":"a7f709b4-e9e2-428f-9c9c-ef397ad87868","Type":"ContainerStarted","Data":"b6a7952b20f25d0df35577ff4e3ee2af2c0179ddc30d1aebc1c6f32dd9c976c5"} Jul 24 13:09:43.526804 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.526779 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" event={"ID":"31c6e268-c2a1-49b3-b910-e185014fcef8","Type":"ContainerStarted","Data":"293c8ba1fded0adae9c84328a892a16718e1192035418df8311fd185739f7dd6"} Jul 24 13:09:43.528971 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.528950 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" event={"ID":"13956c89-0f25-43f9-be04-ab203a8de1f9","Type":"ContainerStarted","Data":"5e6496d0bf2ae91f78d47552f91357eb9fbbe119111b2015dcb43ec99fb68e2a"} Jul 24 13:09:43.530927 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.530899 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nwrp8" event={"ID":"2c393ace-cf46-4319-8dc3-548b7bf80ce0","Type":"ContainerStarted","Data":"344258ba5c6137f913c138eb670e6180bf43732095c50f3fb863eddec5d53c34"} Jul 24 13:09:43.574728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.574643 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" podStartSLOduration=34.812136934 podStartE2EDuration="42.574618334s" totalImagesPullingTime="7.7624814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:43.551144123 +0000 UTC m=+46.915068215" watchObservedRunningTime="2026-07-24 13:09:43.574618334 +0000 UTC m=+46.938542428" Jul 24 13:09:43.575609 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.575570 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-sr5ct" podStartSLOduration=34.878503943 podStartE2EDuration="42.575558077s" totalImagesPullingTime="7.697054134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:43.573437558 +0000 UTC m=+46.937361651" watchObservedRunningTime="2026-07-24 13:09:43.575558077 +0000 UTC m=+46.939482165" Jul 24 13:09:43.601790 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.601715 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-69k7c" podStartSLOduration=34.811146023 podStartE2EDuration="42.601687771s" totalImagesPullingTime="7.790541748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:43.589291198 +0000 UTC m=+46.953215292" watchObservedRunningTime="2026-07-24 13:09:43.601687771 +0000 UTC m=+46.965611863" Jul 24 13:09:43.606173 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.606108 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-59zm2" podStartSLOduration=34.812128692 podStartE2EDuration="42.606084165s" totalImagesPullingTime="7.793955473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:43.604775716 +0000 UTC m=+46.968699808" watchObservedRunningTime="2026-07-24 13:09:43.606084165 +0000 UTC m=+46.970008256" Jul 24 13:09:43.627631 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.627519 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" podStartSLOduration=34.867561215 podStartE2EDuration="42.627502347s" totalImagesPullingTime="7.759941132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:43.627468735 +0000 UTC m=+46.991392828" watchObservedRunningTime="2026-07-24 13:09:43.627502347 +0000 UTC m=+46.991426435" Jul 24 13:09:43.648364 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.648247 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nwrp8" podStartSLOduration=2.889114686 podStartE2EDuration="10.648228846s" totalImagesPullingTime="7.75911416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:43.647092428 +0000 UTC m=+47.011016511" watchObservedRunningTime="2026-07-24 13:09:43.648228846 +0000 UTC m=+47.012152940" Jul 24 13:09:43.678076 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.677981 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:43.778807 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.778674 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:43.924439 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.924284 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:43.924439 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:43.924409 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:44.320576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.320525 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:44.537502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.537469 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:09:44.537914 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.537901 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/0.log" Jul 24 13:09:44.537966 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.537943 2558 generic.go:356] "Generic (PLEG): container finished" podID="279aea8c-817d-4897-b402-9da959d25728" containerID="5709849695bf940d32d0e822ddc84391c47f00b343331a73ee99d9a137787acc" exitCode=255 Jul 24 13:09:44.538074 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.538046 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" event={"ID":"279aea8c-817d-4897-b402-9da959d25728","Type":"ContainerDied","Data":"5709849695bf940d32d0e822ddc84391c47f00b343331a73ee99d9a137787acc"} Jul 24 13:09:44.538203 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.538110 2558 scope.go:117] "RemoveContainer" containerID="d2655f2ed531d42ae2ab9bf3cb83ab0bcbf497c713dfa620cd00cb6be4832803" Jul 24 13:09:44.538373 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.538352 2558 scope.go:117] "RemoveContainer" containerID="5709849695bf940d32d0e822ddc84391c47f00b343331a73ee99d9a137787acc" Jul 24 13:09:44.538600 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:44.538578 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-4sl4t_openshift-console-operator(279aea8c-817d-4897-b402-9da959d25728)\"" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" podUID="279aea8c-817d-4897-b402-9da959d25728" Jul 24 13:09:44.569110 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.569037 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:44.603449 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.603415 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4"] Jul 24 13:09:44.636131 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.636088 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4"] Jul 24 13:09:44.636332 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.636205 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" Jul 24 13:09:44.638967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.638939 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:44.639144 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.639011 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 13:09:44.639144 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.639067 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4kg5x\"" Jul 24 13:09:44.655710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.655675 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:44.720436 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.720378 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:44.736586 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.736552 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbhp9\" (UniqueName: \"kubernetes.io/projected/5f1c92fc-37ef-4c0e-b52b-05a14cfac011-kube-api-access-zbhp9\") pod \"migrator-6865c9967-6qjf4\" (UID: \"5f1c92fc-37ef-4c0e-b52b-05a14cfac011\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" Jul 24 13:09:44.837823 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.837717 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbhp9\" (UniqueName: \"kubernetes.io/projected/5f1c92fc-37ef-4c0e-b52b-05a14cfac011-kube-api-access-zbhp9\") pod \"migrator-6865c9967-6qjf4\" (UID: \"5f1c92fc-37ef-4c0e-b52b-05a14cfac011\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" Jul 24 13:09:44.850906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.850833 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbhp9\" (UniqueName: \"kubernetes.io/projected/5f1c92fc-37ef-4c0e-b52b-05a14cfac011-kube-api-access-zbhp9\") pod \"migrator-6865c9967-6qjf4\" (UID: \"5f1c92fc-37ef-4c0e-b52b-05a14cfac011\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" Jul 24 13:09:44.946825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.946783 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" Jul 24 13:09:44.972143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.972106 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hjt56"] Jul 24 13:09:44.993748 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.993709 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hjt56"] Jul 24 13:09:44.993956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.993806 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:44.997059 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.997029 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 13:09:44.997059 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.997042 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 13:09:44.997604 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:44.997590 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xhz7z\"" Jul 24 13:09:45.029640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.029576 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:45.039702 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.039590 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.039702 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.039657 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa847532-3ffa-44e0-b42a-0968ac99aade-data-volume\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.039912 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.039791 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.039912 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.039864 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa847532-3ffa-44e0-b42a-0968ac99aade-crio-socket\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.039912 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.039899 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cw5p\" (UniqueName: \"kubernetes.io/projected/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-api-access-6cw5p\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.113102 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.112299 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4"] Jul 24 13:09:45.114875 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.114837 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:45.117155 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:45.117130 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1c92fc_37ef_4c0e_b52b_05a14cfac011.slice/crio-f8cfb75f1b87d0dc4c6d92a1982a8eae59f7ea43c47eda764bea85c16f602f83 WatchSource:0}: Error finding container f8cfb75f1b87d0dc4c6d92a1982a8eae59f7ea43c47eda764bea85c16f602f83: Status 404 returned error can't find the container with id f8cfb75f1b87d0dc4c6d92a1982a8eae59f7ea43c47eda764bea85c16f602f83 Jul 24 13:09:45.141298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141255 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa847532-3ffa-44e0-b42a-0968ac99aade-data-volume\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141334 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa847532-3ffa-44e0-b42a-0968ac99aade-crio-socket\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141380 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cw5p\" (UniqueName: \"kubernetes.io/projected/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-api-access-6cw5p\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141510 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.141496 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:09:45.141734 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.141574 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:45.641554478 +0000 UTC m=+49.005478550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : configmap "kube-rbac-proxy" not found Jul 24 13:09:45.141734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141591 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa847532-3ffa-44e0-b42a-0968ac99aade-crio-socket\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141635 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.141687 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa847532-3ffa-44e0-b42a-0968ac99aade-data-volume\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.141734 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.141735 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:09:45.141938 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.141761 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:45.641753872 +0000 UTC m=+49.005677941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : secret "insights-runtime-extractor-tls" not found Jul 24 13:09:45.151672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.151641 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cw5p\" (UniqueName: \"kubernetes.io/projected/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-api-access-6cw5p\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.543616 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.543533 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:09:45.544017 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.543941 2558 scope.go:117] "RemoveContainer" containerID="5709849695bf940d32d0e822ddc84391c47f00b343331a73ee99d9a137787acc" Jul 24 13:09:45.544147 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.544125 2558 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-4sl4t_openshift-console-operator(279aea8c-817d-4897-b402-9da959d25728)\"" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" podUID="279aea8c-817d-4897-b402-9da959d25728" Jul 24 13:09:45.544806 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.544786 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" event={"ID":"5f1c92fc-37ef-4c0e-b52b-05a14cfac011","Type":"ContainerStarted","Data":"f8cfb75f1b87d0dc4c6d92a1982a8eae59f7ea43c47eda764bea85c16f602f83"} Jul 24 13:09:45.645342 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.645300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.645572 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.645475 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:09:45.645572 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:45.645522 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:45.645572 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.645566 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:46.645546593 +0000 UTC m=+50.009470668 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : configmap "kube-rbac-proxy" not found Jul 24 13:09:45.645750 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.645616 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:09:45.645750 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:45.645664 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:46.645649113 +0000 UTC m=+50.009573181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : secret "insights-runtime-extractor-tls" not found Jul 24 13:09:46.619024 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.618988 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-bdbgd"] Jul 24 13:09:46.631526 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.631499 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-bdbgd"] Jul 24 13:09:46.631692 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.631595 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.634212 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.634138 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 13:09:46.634212 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.634209 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 13:09:46.634472 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.634264 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 13:09:46.635220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.635195 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ssstw\"" Jul 24 13:09:46.635346 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.635195 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 13:09:46.656430 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.656366 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:46.656639 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:46.656513 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:09:46.656639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.656526 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:46.656639 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:46.656609 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:48.656586409 +0000 UTC m=+52.020510486 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : configmap "kube-rbac-proxy" not found Jul 24 13:09:46.656826 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:46.656668 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:09:46.656826 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:46.656720 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:48.656707586 +0000 UTC m=+52.020631659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : secret "insights-runtime-extractor-tls" not found Jul 24 13:09:46.734592 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.734553 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:46.757515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.757473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbdg7\" (UniqueName: \"kubernetes.io/projected/f40d45c8-c829-4ea9-9b69-bd5503e512ef-kube-api-access-hbdg7\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.757724 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.757544 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f40d45c8-c829-4ea9-9b69-bd5503e512ef-signing-key\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.757724 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.757684 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f40d45c8-c829-4ea9-9b69-bd5503e512ef-signing-cabundle\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.835927 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.835890 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:46.858783 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.858755 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbdg7\" (UniqueName: \"kubernetes.io/projected/f40d45c8-c829-4ea9-9b69-bd5503e512ef-kube-api-access-hbdg7\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.858945 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.858797 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f40d45c8-c829-4ea9-9b69-bd5503e512ef-signing-key\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.858945 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.858875 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f40d45c8-c829-4ea9-9b69-bd5503e512ef-signing-cabundle\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.859507 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.859490 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f40d45c8-c829-4ea9-9b69-bd5503e512ef-signing-cabundle\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.861181 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.861162 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f40d45c8-c829-4ea9-9b69-bd5503e512ef-signing-key\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.867290 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.867268 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbdg7\" (UniqueName: \"kubernetes.io/projected/f40d45c8-c829-4ea9-9b69-bd5503e512ef-kube-api-access-hbdg7\") pod \"service-ca-658d7b7c78-bdbgd\" (UID: \"f40d45c8-c829-4ea9-9b69-bd5503e512ef\") " pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:46.941913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:46.941880 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" Jul 24 13:09:47.094209 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.094176 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-bdbgd"] Jul 24 13:09:47.097411 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:09:47.097371 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40d45c8_c829_4ea9_9b69_bd5503e512ef.slice/crio-2b7b71726d76c5d5bc7de9370f164a75fb905cc12fe32b689be0e3334f60c9fd WatchSource:0}: Error finding container 2b7b71726d76c5d5bc7de9370f164a75fb905cc12fe32b689be0e3334f60c9fd: Status 404 returned error can't find the container with id 2b7b71726d76c5d5bc7de9370f164a75fb905cc12fe32b689be0e3334f60c9fd Jul 24 13:09:47.166960 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.166929 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zm4hb_077ed00b-9339-4528-948b-daf1bebab5fc/dns-node-resolver/0.log" Jul 24 13:09:47.553473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.553346 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" event={"ID":"f40d45c8-c829-4ea9-9b69-bd5503e512ef","Type":"ContainerStarted","Data":"0418763838ecc7575de47c102a5ad103de2ec819751256fbe4f5d9322d036ff6"} Jul 24 13:09:47.553473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.553426 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" event={"ID":"f40d45c8-c829-4ea9-9b69-bd5503e512ef","Type":"ContainerStarted","Data":"2b7b71726d76c5d5bc7de9370f164a75fb905cc12fe32b689be0e3334f60c9fd"} Jul 24 13:09:47.555215 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.555188 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" event={"ID":"5f1c92fc-37ef-4c0e-b52b-05a14cfac011","Type":"ContainerStarted","Data":"ebe4edb7982efaa5df1ed58cd303cd81f42cd5eb68b824879da6d9bbf9ab6119"} Jul 24 13:09:47.555344 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.555242 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" event={"ID":"5f1c92fc-37ef-4c0e-b52b-05a14cfac011","Type":"ContainerStarted","Data":"8564f6e816b6ce09bb635d88d28e9f5745edfeda7ac85c9cab6f244f8c6584f6"} Jul 24 13:09:47.572778 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.572715 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-bdbgd" podStartSLOduration=1.5726946819999998 podStartE2EDuration="1.572694682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:47.571752337 +0000 UTC m=+50.935676431" watchObservedRunningTime="2026-07-24 13:09:47.572694682 +0000 UTC m=+50.936618773" Jul 24 13:09:47.590589 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.590529 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-6qjf4" podStartSLOduration=1.893004702 podStartE2EDuration="3.590512089s" totalImagesPullingTime="1.697507387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:47.590005463 +0000 UTC m=+50.953929555" watchObservedRunningTime="2026-07-24 13:09:47.590512089 +0000 UTC m=+50.954436179" Jul 24 13:09:47.967343 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:47.967312 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87xcs_595f165a-81a3-4662-9fb1-6e5e7c7dacb9/node-ca/0.log" Jul 24 13:09:48.297119 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.297017 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:48.399243 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.398965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:48.499957 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.499923 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:48.567195 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.567107 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwrp8_2c393ace-cf46-4319-8dc3-548b7bf80ce0/serve-healthcheck-canary/0.log" Jul 24 13:09:48.609728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.609694 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:48.677662 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.677619 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:48.677826 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:48.677750 2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:09:48.677826 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:48.677817 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:52.677798728 +0000 UTC m=+56.041722810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : configmap "kube-rbac-proxy" not found Jul 24 13:09:48.677951 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.677841 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:48.678014 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:48.677971 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:09:48.678060 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:48.678044 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:09:52.678023786 +0000 UTC m=+56.041947874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : secret "insights-runtime-extractor-tls" not found Jul 24 13:09:48.699722 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.699680 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:48.769915 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.769874 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-6qjf4_5f1c92fc-37ef-4c0e-b52b-05a14cfac011/migrator/0.log" Jul 24 13:09:48.800260 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.800217 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:48.967288 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:48.967257 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-6qjf4_5f1c92fc-37ef-4c0e-b52b-05a14cfac011/graceful-termination/0.log" Jul 24 13:09:49.007093 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.007054 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:49.105802 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.105753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:49.168974 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.168931 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh_13956c89-0f25-43f9-be04-ab203a8de1f9/kube-storage-version-migrator-operator/0.log" Jul 24 13:09:49.485157 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485121 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485183 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485214 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485231 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485247 2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485317 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485338 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 13:09:49.485354 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485353 2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485346 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.485322742 +0000 UTC m=+68.849246814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : secret "router-metrics-certs-default" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485434 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert podName:0825d55b-6343-411d-8a8c-9dcc1d46bf93 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.48541854 +0000 UTC m=+68.849342613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-5vfs6" (UID: "0825d55b-6343-411d-8a8c-9dcc1d46bf93") : secret "networking-console-plugin-cert" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485451 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle podName:6e76b0c5-9196-42ef-8515-86b4ec2d1f86 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.485442943 +0000 UTC m=+68.849367011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle") pod "router-default-6b84687b8c-kmckn" (UID: "6e76b0c5-9196-42ef-8515-86b4ec2d1f86") : configmap references non-existent config key: service-ca.crt Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485466 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls podName:9b3afa1c-c462-4ce1-891c-7412da5b4566 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.485458304 +0000 UTC m=+68.849382373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-sbzh4" (UID: "9b3afa1c-c462-4ce1-891c-7412da5b4566") : secret "cluster-monitoring-operator-tls" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485475 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485528 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls podName:b5db3487-f9e9-412c-a5b3-d4e661de0280 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.485511517 +0000 UTC m=+68.849435588 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls") pod "dns-default-z9vkb" (UID: "b5db3487-f9e9-412c-a5b3-d4e661de0280") : secret "dns-default-metrics-tls" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.485572 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485690 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485703 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57bf4c6dfb-zscvm: secret "image-registry-tls" not found Jul 24 13:09:49.485814 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.485749 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls podName:253c7174-7697-4369-85c6-b0f3abe58c64 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.485737619 +0000 UTC m=+68.849661690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls") pod "image-registry-57bf4c6dfb-zscvm" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64") : secret "image-registry-tls" not found Jul 24 13:09:49.487737 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.487706 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d842ba9-bc55-4f50-a760-399f427bb42d-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-rsmhc\" (UID: \"3d842ba9-bc55-4f50-a760-399f427bb42d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:49.587164 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.587114 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:09:49.587417 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.587270 2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 13:09:49.587417 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.587289 2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-86db5847b7-cbr7q: secret "image-registry-tls" not found Jul 24 13:09:49.587417 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:49.587350 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls podName:d0c4d641-690d-4043-830f-9b2dfd0dabbb nodeName:}" failed. No retries permitted until 2026-07-24 13:10:05.587332004 +0000 UTC m=+68.951256078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls") pod "image-registry-86db5847b7-cbr7q" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb") : secret "image-registry-tls" not found Jul 24 13:09:49.754017 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.753930 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" Jul 24 13:09:49.903894 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.903856 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc"] Jul 24 13:09:49.988575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:49.988544 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:50.404735 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:50.404702 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:50.570005 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:50.569960 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" event={"ID":"3d842ba9-bc55-4f50-a760-399f427bb42d","Type":"ContainerStarted","Data":"4020541cf7370f27c4591b495f3cb1bd59592b15d7f857e30cb5cd3f6ed33a57"} Jul 24 13:09:50.616282 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:50.616134 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:50.716733 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:50.716616 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:50.816925 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:50.816888 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:50.989191 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:50.989081 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:51.680313 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:51.680258 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:51.981250 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:51.981145 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:52.081518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.081483 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:52.418241 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.418198 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:52.583035 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.582998 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" event={"ID":"3d842ba9-bc55-4f50-a760-399f427bb42d","Type":"ContainerStarted","Data":"2bbab1d83fd895beb26adf23bc2ab543148de875dd7500755eb81e11572f0128"} Jul 24 13:09:52.583221 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.583084 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" event={"ID":"3d842ba9-bc55-4f50-a760-399f427bb42d","Type":"ContainerStarted","Data":"1d659cd2b608aea1dc9071229fbb304938b66df63f1af7dba8408b57060b448a"} Jul 24 13:09:52.599296 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.599250 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-rsmhc" podStartSLOduration=49.599691049 podStartE2EDuration="51.599227252s" totalImagesPullingTime="1.999536203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:52.599052357 +0000 UTC m=+55.962976450" watchObservedRunningTime="2026-07-24 13:09:52.599227252 +0000 UTC m=+55.963151321" Jul 24 13:09:52.608716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.608660 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:52.720448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.720335 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:52.720795 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.720493 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:52.720795 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:52.720499 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:09:52.720795 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:09:52.720638 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls podName:aa847532-3ffa-44e0-b42a-0968ac99aade nodeName:}" failed. No retries permitted until 2026-07-24 13:10:00.720618343 +0000 UTC m=+64.084542430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hjt56" (UID: "aa847532-3ffa-44e0-b42a-0968ac99aade") : secret "insights-runtime-extractor-tls" not found Jul 24 13:09:52.721053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.721031 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa847532-3ffa-44e0-b42a-0968ac99aade-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:09:52.791504 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:52.791458 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:53.191106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:53.191058 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:53.599756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:53.599727 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:53.924527 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:53.924437 2558 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:53.924527 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:53.924507 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:53.924998 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:53.924883 2558 scope.go:117] "RemoveContainer" containerID="5709849695bf940d32d0e822ddc84391c47f00b343331a73ee99d9a137787acc" Jul 24 13:09:53.978286 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:53.978247 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:54.189818 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.189726 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:54.585571 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.585441 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:54.593302 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.593273 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:09:54.593499 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.593341 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" event={"ID":"279aea8c-817d-4897-b402-9da959d25728","Type":"ContainerStarted","Data":"dcd1c68941857913c6ba42bdcbd3ff399657d6a5df3b71a67203c54b8c26559b"} Jul 24 13:09:54.593818 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.593796 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:54.613351 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.613284 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" podStartSLOduration=45.847464298 podStartE2EDuration="53.613260326s" totalImagesPullingTime="7.765796028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:54.612208013 +0000 UTC m=+57.976132105" watchObservedRunningTime="2026-07-24 13:09:54.613260326 +0000 UTC m=+57.977184418" Jul 24 13:09:54.644833 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.644800 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-4sl4t" Jul 24 13:09:54.645676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.645649 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:54.795085 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.795040 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:54.896687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.896643 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:54.992317 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:54.992280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:55.114907 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:55.114865 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:55.216868 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:55.216747 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:55.513208 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:55.513113 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:55.614087 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:55.614046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:55.713714 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:55.713673 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:55.918367 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:55.918327 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.022745 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.022699 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.108917 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.108872 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.173665 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.173577 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.387963 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.387931 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.510681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.510579 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.611542 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.611502 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.720570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.720535 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:56.821473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:56.821363 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:57.188490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:57.188449 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:57.446849 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:57.446765 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2bvgd" Jul 24 13:09:57.758080 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:57.757971 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:57.937722 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:57.937671 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:58.038686 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:58.038585 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:58.401116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:58.401070 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:58.596325 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:58.596289 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:58.816747 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:58.816644 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:59.002119 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:59.002078 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:09:59.390038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:09:59.389996 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:00.805366 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:00.805323 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:10:00.807715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:00.807691 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa847532-3ffa-44e0-b42a-0968ac99aade-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hjt56\" (UID: \"aa847532-3ffa-44e0-b42a-0968ac99aade\") " pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:10:00.906915 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:00.906881 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xhz7z\"" Jul 24 13:10:00.915004 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:00.914968 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hjt56" Jul 24 13:10:01.050729 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:01.050683 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hjt56"] Jul 24 13:10:01.054040 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:01.054010 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa847532_3ffa_44e0_b42a_0968ac99aade.slice/crio-1a4591696032771993020ddae86a11fedc9888911c0930a1dc02f7afd9e0033a WatchSource:0}: Error finding container 1a4591696032771993020ddae86a11fedc9888911c0930a1dc02f7afd9e0033a: Status 404 returned error can't find the container with id 1a4591696032771993020ddae86a11fedc9888911c0930a1dc02f7afd9e0033a Jul 24 13:10:01.617174 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:01.617134 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjt56" event={"ID":"aa847532-3ffa-44e0-b42a-0968ac99aade","Type":"ContainerStarted","Data":"1a4591696032771993020ddae86a11fedc9888911c0930a1dc02f7afd9e0033a"} Jul 24 13:10:02.822761 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.822721 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:10:02.829416 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.825717 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:10:02.835714 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.835682 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdb9d81c-3072-4b3b-a7f8-e6bf692e622e-metrics-certs\") pod \"network-metrics-daemon-pnd68\" (UID: \"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e\") " pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:10:02.923892 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.923850 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:10:02.924078 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.923938 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:10:02.926992 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.926964 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:10:02.926992 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.926973 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fs4g\" (UniqueName: \"kubernetes.io/projected/46e66420-02a4-4eb9-83ba-869350f09cc1-kube-api-access-7fs4g\") pod \"network-check-target-sdkhm\" (UID: \"46e66420-02a4-4eb9-83ba-869350f09cc1\") " pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:10:02.937257 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:02.937221 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4971787d-2c92-40a4-94d9-6ab28883e02b-original-pull-secret\") pod \"global-pull-secret-syncer-8jsc6\" (UID: \"4971787d-2c92-40a4-94d9-6ab28883e02b\") " pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:10:03.103861 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.103824 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rh25q\"" Jul 24 13:10:03.109299 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.109265 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dqwnd\"" Jul 24 13:10:03.111012 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.110986 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pnd68" Jul 24 13:10:03.112131 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.112104 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jsc6" Jul 24 13:10:03.117376 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.117345 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:10:03.302254 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.302218 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdkhm"] Jul 24 13:10:03.308223 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:03.308155 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e66420_02a4_4eb9_83ba_869350f09cc1.slice/crio-dfcb9a45141ad59208a0ac09563d8e2e933d04727328f659f6fc9b7f72679ad3 WatchSource:0}: Error finding container dfcb9a45141ad59208a0ac09563d8e2e933d04727328f659f6fc9b7f72679ad3: Status 404 returned error can't find the container with id dfcb9a45141ad59208a0ac09563d8e2e933d04727328f659f6fc9b7f72679ad3 Jul 24 13:10:03.326171 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:03.326124 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4971787d_2c92_40a4_94d9_6ab28883e02b.slice/crio-27ccb2c038da6b2d2844b13de9027eb6df1e936f3ad0ce5a0a17f889f444ca73 WatchSource:0}: Error finding container 27ccb2c038da6b2d2844b13de9027eb6df1e936f3ad0ce5a0a17f889f444ca73: Status 404 returned error can't find the container with id 27ccb2c038da6b2d2844b13de9027eb6df1e936f3ad0ce5a0a17f889f444ca73 Jul 24 13:10:03.328002 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:03.327467 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb9d81c_3072_4b3b_a7f8_e6bf692e622e.slice/crio-a182bc4d4c20b18f8f3a2749e854359537a9b313f6d543eef42536448556bd10 WatchSource:0}: Error finding container a182bc4d4c20b18f8f3a2749e854359537a9b313f6d543eef42536448556bd10: Status 404 returned error can't find the container with id a182bc4d4c20b18f8f3a2749e854359537a9b313f6d543eef42536448556bd10 Jul 24 13:10:03.329099 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.329018 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8jsc6"] Jul 24 13:10:03.330885 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.330340 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pnd68"] Jul 24 13:10:03.627552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.627447 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8jsc6" event={"ID":"4971787d-2c92-40a4-94d9-6ab28883e02b","Type":"ContainerStarted","Data":"27ccb2c038da6b2d2844b13de9027eb6df1e936f3ad0ce5a0a17f889f444ca73"} Jul 24 13:10:03.628941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.628902 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pnd68" event={"ID":"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e","Type":"ContainerStarted","Data":"a182bc4d4c20b18f8f3a2749e854359537a9b313f6d543eef42536448556bd10"} Jul 24 13:10:03.630757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.630716 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdkhm" event={"ID":"46e66420-02a4-4eb9-83ba-869350f09cc1","Type":"ContainerStarted","Data":"5c66f09e3a1d1acf117bc8fe23b35da1b72129a76638fe9100d994ac47ac0eb8"} Jul 24 13:10:03.630906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.630769 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdkhm" event={"ID":"46e66420-02a4-4eb9-83ba-869350f09cc1","Type":"ContainerStarted","Data":"dfcb9a45141ad59208a0ac09563d8e2e933d04727328f659f6fc9b7f72679ad3"} Jul 24 13:10:03.630906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.630807 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:10:03.649900 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:03.649832 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sdkhm" podStartSLOduration=65.649806373 podStartE2EDuration="1m5.649806373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:03.648294919 +0000 UTC m=+67.012219011" watchObservedRunningTime="2026-07-24 13:10:03.649806373 +0000 UTC m=+67.013730465" Jul 24 13:10:05.550408 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.550350 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:10:05.551011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.550420 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:10:05.551011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.550462 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:10:05.551011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.550502 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:05.551011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.550556 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:10:05.551011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.550750 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:05.551422 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.551339 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-service-ca-bundle\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:05.553622 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.553584 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b5db3487-f9e9-412c-a5b3-d4e661de0280-metrics-tls\") pod \"dns-default-z9vkb\" (UID: \"b5db3487-f9e9-412c-a5b3-d4e661de0280\") " pod="openshift-dns/dns-default-z9vkb" Jul 24 13:10:05.553777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.553599 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e76b0c5-9196-42ef-8515-86b4ec2d1f86-metrics-certs\") pod \"router-default-6b84687b8c-kmckn\" (UID: \"6e76b0c5-9196-42ef-8515-86b4ec2d1f86\") " pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:05.553777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.553650 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"image-registry-57bf4c6dfb-zscvm\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:10:05.553777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.553696 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0825d55b-6343-411d-8a8c-9dcc1d46bf93-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-5vfs6\" (UID: \"0825d55b-6343-411d-8a8c-9dcc1d46bf93\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:10:05.553981 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.553957 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9b3afa1c-c462-4ce1-891c-7412da5b4566-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-sbzh4\" (UID: \"9b3afa1c-c462-4ce1-891c-7412da5b4566\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:10:05.652123 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.652074 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:05.654597 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.654567 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"image-registry-86db5847b7-cbr7q\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:05.680322 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.680289 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kvvpn\"" Jul 24 13:10:05.688397 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.688355 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:05.703738 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.703707 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cfzrs\"" Jul 24 13:10:05.709172 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.709151 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-sz6sn\"" Jul 24 13:10:05.711217 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.711202 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" Jul 24 13:10:05.714502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.714480 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:10:05.717561 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.717538 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:05.741597 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.741572 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmgtg\"" Jul 24 13:10:05.749186 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.749149 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" Jul 24 13:10:05.772398 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.772370 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n54jn\"" Jul 24 13:10:05.780511 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:05.780482 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z9vkb" Jul 24 13:10:06.550897 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.550864 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx"] Jul 24 13:10:06.553467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.553436 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.557805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.557540 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 13:10:06.557805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.557704 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 24 13:10:06.558046 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.557990 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 13:10:06.558272 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.558241 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 13:10:06.560518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.560489 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r"] Jul 24 13:10:06.563922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.563760 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6"] Jul 24 13:10:06.565125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.565105 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.568019 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.567999 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx"] Jul 24 13:10:06.568233 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.568207 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.571108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.571083 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 24 13:10:06.571878 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.571836 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 13:10:06.572770 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.572739 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 13:10:06.573121 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.573098 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 13:10:06.578409 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.574543 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 13:10:06.578726 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.578669 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r"] Jul 24 13:10:06.581415 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.581007 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gc2gg\"" Jul 24 13:10:06.584347 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.584320 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6"] Jul 24 13:10:06.634288 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.634229 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57bf4c6dfb-zscvm"] Jul 24 13:10:06.658299 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.658257 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-fv88n"] Jul 24 13:10:06.661015 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.660984 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:06.661462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661281 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7956822-ac01-4217-89c1-dfb2fdc18fad-tmp\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.661462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.661462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661346 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-ca\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.661462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661372 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e8351b-ad82-4642-b460-974e4f10793f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.661462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661435 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7956822-ac01-4217-89c1-dfb2fdc18fad-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.661797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661507 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r5p2\" (UniqueName: \"kubernetes.io/projected/e983a1fa-d21b-41fc-933d-7ab33bdb5064-kube-api-access-5r5p2\") pod \"managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r\" (UID: \"e983a1fa-d21b-41fc-933d-7ab33bdb5064\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.661797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661545 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-hub\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.661797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp8w7\" (UniqueName: \"kubernetes.io/projected/e7956822-ac01-4217-89c1-dfb2fdc18fad-kube-api-access-rp8w7\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.661797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks25k\" (UniqueName: \"kubernetes.io/projected/72e8351b-ad82-4642-b460-974e4f10793f-kube-api-access-ks25k\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.661797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661660 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.661797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.661698 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e983a1fa-d21b-41fc-933d-7ab33bdb5064-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r\" (UID: \"e983a1fa-d21b-41fc-933d-7ab33bdb5064\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.663971 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.663846 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 13:10:06.663971 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.663932 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 13:10:06.664265 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.664174 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ktb7z\"" Jul 24 13:10:06.681957 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.681924 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-fv88n"] Jul 24 13:10:06.763270 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763229 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqxxg\" (UniqueName: \"kubernetes.io/projected/731228c1-003d-4c8b-9f37-2caab3adf288-kube-api-access-sqxxg\") pod \"downloads-7c97dc4555-fv88n\" (UID: \"731228c1-003d-4c8b-9f37-2caab3adf288\") " pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:06.763476 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7956822-ac01-4217-89c1-dfb2fdc18fad-tmp\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.763476 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.763613 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763499 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-ca\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.763613 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763546 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e8351b-ad82-4642-b460-974e4f10793f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.763613 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763581 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7956822-ac01-4217-89c1-dfb2fdc18fad-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.763762 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763634 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r5p2\" (UniqueName: \"kubernetes.io/projected/e983a1fa-d21b-41fc-933d-7ab33bdb5064-kube-api-access-5r5p2\") pod \"managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r\" (UID: \"e983a1fa-d21b-41fc-933d-7ab33bdb5064\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.763762 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763665 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-hub\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.763762 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763693 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp8w7\" (UniqueName: \"kubernetes.io/projected/e7956822-ac01-4217-89c1-dfb2fdc18fad-kube-api-access-rp8w7\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.763762 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763739 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks25k\" (UniqueName: \"kubernetes.io/projected/72e8351b-ad82-4642-b460-974e4f10793f-kube-api-access-ks25k\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.763956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763778 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.763956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763787 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7956822-ac01-4217-89c1-dfb2fdc18fad-tmp\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.763956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.763817 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e983a1fa-d21b-41fc-933d-7ab33bdb5064-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r\" (UID: \"e983a1fa-d21b-41fc-933d-7ab33bdb5064\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.764822 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.764266 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72e8351b-ad82-4642-b460-974e4f10793f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.768123 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.768072 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-ca\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.768254 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.768119 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-hub\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.769588 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.769521 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.769917 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.769889 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72e8351b-ad82-4642-b460-974e4f10793f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.770303 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.770278 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e7956822-ac01-4217-89c1-dfb2fdc18fad-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.771881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.771858 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e983a1fa-d21b-41fc-933d-7ab33bdb5064-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r\" (UID: \"e983a1fa-d21b-41fc-933d-7ab33bdb5064\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.779075 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.779016 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r5p2\" (UniqueName: \"kubernetes.io/projected/e983a1fa-d21b-41fc-933d-7ab33bdb5064-kube-api-access-5r5p2\") pod \"managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r\" (UID: \"e983a1fa-d21b-41fc-933d-7ab33bdb5064\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.780696 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.780668 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp8w7\" (UniqueName: \"kubernetes.io/projected/e7956822-ac01-4217-89c1-dfb2fdc18fad-kube-api-access-rp8w7\") pod \"klusterlet-addon-workmgr-6d6ffd6fff-mz9fx\" (UID: \"e7956822-ac01-4217-89c1-dfb2fdc18fad\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.781100 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.781077 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks25k\" (UniqueName: \"kubernetes.io/projected/72e8351b-ad82-4642-b460-974e4f10793f-kube-api-access-ks25k\") pod \"cluster-proxy-proxy-agent-78cdd7587f-ptkp6\" (UID: \"72e8351b-ad82-4642-b460-974e4f10793f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.864655 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.864615 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqxxg\" (UniqueName: \"kubernetes.io/projected/731228c1-003d-4c8b-9f37-2caab3adf288-kube-api-access-sqxxg\") pod \"downloads-7c97dc4555-fv88n\" (UID: \"731228c1-003d-4c8b-9f37-2caab3adf288\") " pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:06.873339 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.873272 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:06.874108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.874084 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqxxg\" (UniqueName: \"kubernetes.io/projected/731228c1-003d-4c8b-9f37-2caab3adf288-kube-api-access-sqxxg\") pod \"downloads-7c97dc4555-fv88n\" (UID: \"731228c1-003d-4c8b-9f37-2caab3adf288\") " pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:06.905739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.905488 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" Jul 24 13:10:06.908969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.908919 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:06.912735 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.912633 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" Jul 24 13:10:06.982662 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:06.982606 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:07.009522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.009473 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:07.047885 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.047803 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b84687b8c-kmckn"] Jul 24 13:10:07.112839 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.112801 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:07.154720 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:07.154684 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e76b0c5_9196_42ef_8515_86b4ec2d1f86.slice/crio-adba3ada1378853f6e08359f817ab4725b592d9ac2d5e9c1afee5137dcb440d3 WatchSource:0}: Error finding container adba3ada1378853f6e08359f817ab4725b592d9ac2d5e9c1afee5137dcb440d3: Status 404 returned error can't find the container with id adba3ada1378853f6e08359f817ab4725b592d9ac2d5e9c1afee5137dcb440d3 Jul 24 13:10:07.280779 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.280638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:07.417329 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.416254 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6"] Jul 24 13:10:07.462607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.462573 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z9vkb"] Jul 24 13:10:07.480436 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.480204 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57bf4c6dfb-zscvm"] Jul 24 13:10:07.487867 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:07.487828 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253c7174_7697_4369_85c6_b0f3abe58c64.slice/crio-e701df91b8bc231d3823c016dcd443a4c9fc60a06405ad895c78c56b21d91685 WatchSource:0}: Error finding container e701df91b8bc231d3823c016dcd443a4c9fc60a06405ad895c78c56b21d91685: Status 404 returned error can't find the container with id e701df91b8bc231d3823c016dcd443a4c9fc60a06405ad895c78c56b21d91685 Jul 24 13:10:07.555228 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.554538 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4"] Jul 24 13:10:07.591995 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.591871 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx"] Jul 24 13:10:07.654496 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.654307 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjt56" event={"ID":"aa847532-3ffa-44e0-b42a-0968ac99aade","Type":"ContainerStarted","Data":"c8ae30f5dd5b056f3071936f66711d3fd9da2ed6e446e88863a7e0b3498e2592"} Jul 24 13:10:07.657113 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.656764 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" event={"ID":"253c7174-7697-4369-85c6-b0f3abe58c64","Type":"ContainerStarted","Data":"7aea9b43cf089b618a0fc813ce68a981d31332f6d98859f18053898e33b9f630"} Jul 24 13:10:07.657113 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.656811 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" event={"ID":"253c7174-7697-4369-85c6-b0f3abe58c64","Type":"ContainerStarted","Data":"e701df91b8bc231d3823c016dcd443a4c9fc60a06405ad895c78c56b21d91685"} Jul 24 13:10:07.657113 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.656830 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:10:07.658794 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.658734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" event={"ID":"0825d55b-6343-411d-8a8c-9dcc1d46bf93","Type":"ContainerStarted","Data":"47f18dc7d30d37221fd439daa4d32ba6f60b71cc70db85203f79820e78214fa1"} Jul 24 13:10:07.661234 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.661214 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b84687b8c-kmckn" event={"ID":"6e76b0c5-9196-42ef-8515-86b4ec2d1f86","Type":"ContainerStarted","Data":"17c4899bdda58e3094b69881e18d11aa5e86156b2fac8c09713c9f9a9f26b460"} Jul 24 13:10:07.661298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.661254 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b84687b8c-kmckn" event={"ID":"6e76b0c5-9196-42ef-8515-86b4ec2d1f86","Type":"ContainerStarted","Data":"adba3ada1378853f6e08359f817ab4725b592d9ac2d5e9c1afee5137dcb440d3"} Jul 24 13:10:07.664815 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.664716 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pnd68" event={"ID":"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e","Type":"ContainerStarted","Data":"f10ba23aff8578ce92981ed149d145e48704276224ed7576f71041c9962a1997"} Jul 24 13:10:07.667136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.667022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" event={"ID":"e7956822-ac01-4217-89c1-dfb2fdc18fad","Type":"ContainerStarted","Data":"c0a717569d81350ff4888e2c30257f50c3f7bda2a40644dcd04c182370eabf44"} Jul 24 13:10:07.671960 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.671928 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z9vkb" event={"ID":"b5db3487-f9e9-412c-a5b3-d4e661de0280","Type":"ContainerStarted","Data":"a1303824d95173b2f636f06781e1068511d1cf7f084d91fb1f88e1e4fd197dac"} Jul 24 13:10:07.674769 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.674732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" event={"ID":"9b3afa1c-c462-4ce1-891c-7412da5b4566","Type":"ContainerStarted","Data":"125a714fdf826524eb77e11b39100f16871e6d33624801917931a25790ef3e40"} Jul 24 13:10:07.677821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.677790 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-fv88n"] Jul 24 13:10:07.681264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.681190 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" podStartSLOduration=66.681169308 podStartE2EDuration="1m6.681169308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:07.679855491 +0000 UTC m=+71.043779582" watchObservedRunningTime="2026-07-24 13:10:07.681169308 +0000 UTC m=+71.045093401" Jul 24 13:10:07.683993 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:07.683966 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731228c1_003d_4c8b_9f37_2caab3adf288.slice/crio-8aa37531fc718d844b55397e8d64ec493c5de9cbac463990f36a9338b613f7a7 WatchSource:0}: Error finding container 8aa37531fc718d844b55397e8d64ec493c5de9cbac463990f36a9338b613f7a7: Status 404 returned error can't find the container with id 8aa37531fc718d844b55397e8d64ec493c5de9cbac463990f36a9338b613f7a7 Jul 24 13:10:07.702645 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.701749 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6b84687b8c-kmckn" podStartSLOduration=66.701725713 podStartE2EDuration="1m6.701725713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:07.701593776 +0000 UTC m=+71.065517867" watchObservedRunningTime="2026-07-24 13:10:07.701725713 +0000 UTC m=+71.065649805" Jul 24 13:10:07.718513 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.718480 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:07.722531 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.722094 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:07.799519 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.799486 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6"] Jul 24 13:10:07.802994 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.802917 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86db5847b7-cbr7q"] Jul 24 13:10:07.804683 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:07.804481 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r"] Jul 24 13:10:07.806135 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:07.806097 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72e8351b_ad82_4642_b460_974e4f10793f.slice/crio-cc6153b1472e4f7c063f0d5bf623d8593c56c5bf2d120c07251168f4243bff03 WatchSource:0}: Error finding container cc6153b1472e4f7c063f0d5bf623d8593c56c5bf2d120c07251168f4243bff03: Status 404 returned error can't find the container with id cc6153b1472e4f7c063f0d5bf623d8593c56c5bf2d120c07251168f4243bff03 Jul 24 13:10:07.807830 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:07.807793 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode983a1fa_d21b_41fc_933d_7ab33bdb5064.slice/crio-a2aef0ebe2d44e038032bfca74eaf1d137c451f8c08736e303776b2e40310ce9 WatchSource:0}: Error finding container a2aef0ebe2d44e038032bfca74eaf1d137c451f8c08736e303776b2e40310ce9: Status 404 returned error can't find the container with id a2aef0ebe2d44e038032bfca74eaf1d137c451f8c08736e303776b2e40310ce9 Jul 24 13:10:07.810233 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:07.810191 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c4d641_690d_4043_830f_9b2dfd0dabbb.slice/crio-0dcedcc427c9ac67791a85d839f52f9d84fbfb593e601ab69bb583c425a34992 WatchSource:0}: Error finding container 0dcedcc427c9ac67791a85d839f52f9d84fbfb593e601ab69bb583c425a34992: Status 404 returned error can't find the container with id 0dcedcc427c9ac67791a85d839f52f9d84fbfb593e601ab69bb583c425a34992 Jul 24 13:10:08.696867 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.694314 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" event={"ID":"d0c4d641-690d-4043-830f-9b2dfd0dabbb","Type":"ContainerStarted","Data":"5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6"} Jul 24 13:10:08.696867 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.694412 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" event={"ID":"d0c4d641-690d-4043-830f-9b2dfd0dabbb","Type":"ContainerStarted","Data":"0dcedcc427c9ac67791a85d839f52f9d84fbfb593e601ab69bb583c425a34992"} Jul 24 13:10:08.696867 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.694468 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:08.697498 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.696878 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" event={"ID":"e983a1fa-d21b-41fc-933d-7ab33bdb5064","Type":"ContainerStarted","Data":"a2aef0ebe2d44e038032bfca74eaf1d137c451f8c08736e303776b2e40310ce9"} Jul 24 13:10:08.700118 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.700022 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" event={"ID":"72e8351b-ad82-4642-b460-974e4f10793f","Type":"ContainerStarted","Data":"cc6153b1472e4f7c063f0d5bf623d8593c56c5bf2d120c07251168f4243bff03"} Jul 24 13:10:08.705997 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.705930 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-fv88n" event={"ID":"731228c1-003d-4c8b-9f37-2caab3adf288","Type":"ContainerStarted","Data":"8aa37531fc718d844b55397e8d64ec493c5de9cbac463990f36a9338b613f7a7"} Jul 24 13:10:08.716525 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.715553 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" podStartSLOduration=70.715529209 podStartE2EDuration="1m10.715529209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:08.713886686 +0000 UTC m=+72.077810778" watchObservedRunningTime="2026-07-24 13:10:08.715529209 +0000 UTC m=+72.079453302" Jul 24 13:10:08.720727 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.720572 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pnd68" event={"ID":"bdb9d81c-3072-4b3b-a7f8-e6bf692e622e","Type":"ContainerStarted","Data":"3a960860a485f345b0694a4fa57071158f7e7158f12e295540b7f829058ddb28"} Jul 24 13:10:08.720727 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.720632 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:08.722505 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.722463 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6b84687b8c-kmckn" Jul 24 13:10:08.737826 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.737619 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pnd68" podStartSLOduration=66.849833867 podStartE2EDuration="1m10.737594215s" totalImagesPullingTime="3.887760348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:08.735703617 +0000 UTC m=+72.099627709" watchObservedRunningTime="2026-07-24 13:10:08.737594215 +0000 UTC m=+72.101518309" Jul 24 13:10:08.856663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.856594 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:08.917820 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:08.917545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:10.556216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.556169 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:10.599548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.599507 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56dc49f854-7xh74"] Jul 24 13:10:10.609630 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.608084 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56dc49f854-7xh74"] Jul 24 13:10:10.609630 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.608212 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.616279 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.614287 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 13:10:10.616279 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.614573 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z656h\"" Jul 24 13:10:10.616279 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.614815 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 13:10:10.616279 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.614286 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 13:10:10.616279 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.615255 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 13:10:10.626899 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.615374 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 13:10:10.643991 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.643953 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:10.713515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.713480 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-oauth-config\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.713728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.713557 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbccl\" (UniqueName: \"kubernetes.io/projected/b665f366-5864-4d11-90bd-5020a27f66ab-kube-api-access-kbccl\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.713728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.713591 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-service-ca\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.713728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.713621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-serving-cert\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.713728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.713674 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-console-config\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.713728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.713697 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-oauth-serving-cert\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.716256 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.716131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815016 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-console-config\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815067 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-oauth-serving-cert\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815160 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-oauth-config\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815224 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbccl\" (UniqueName: \"kubernetes.io/projected/b665f366-5864-4d11-90bd-5020a27f66ab-kube-api-access-kbccl\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815271 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-service-ca\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815307 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-serving-cert\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.815955 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-console-config\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.816175 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-oauth-serving-cert\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.817405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.817263 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-service-ca\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.840021 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.839949 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-oauth-config\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.840239 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.840183 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-serving-cert\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.843174 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.843138 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbccl\" (UniqueName: \"kubernetes.io/projected/b665f366-5864-4d11-90bd-5020a27f66ab-kube-api-access-kbccl\") pod \"console-56dc49f854-7xh74\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:10.932967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:10.932893 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:11.145606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:11.145539 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:11.209106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:11.209035 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:12.149965 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:12.149926 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:12.249570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:12.249474 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:12.351941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:12.351899 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:12.591829 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:12.591744 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:12.692322 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:12.692253 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:13.057902 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:13.057836 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:13.949213 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:13.949170 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:14.015961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:14.015923 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:15.716024 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:15.715988 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56dc49f854-7xh74"] Jul 24 13:10:16.253826 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:16.253786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:16.482867 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:16.482829 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:16.549282 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:16.549193 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:18.301224 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:18.301191 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb665f366_5864_4d11_90bd_5020a27f66ab.slice/crio-6afe0dcc554334121227350510355ef7a2a8f4ddc6eec04449beb7abeadb2821 WatchSource:0}: Error finding container 6afe0dcc554334121227350510355ef7a2a8f4ddc6eec04449beb7abeadb2821: Status 404 returned error can't find the container with id 6afe0dcc554334121227350510355ef7a2a8f4ddc6eec04449beb7abeadb2821 Jul 24 13:10:18.798789 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:18.798746 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56dc49f854-7xh74" event={"ID":"b665f366-5864-4d11-90bd-5020a27f66ab","Type":"ContainerStarted","Data":"6afe0dcc554334121227350510355ef7a2a8f4ddc6eec04449beb7abeadb2821"} Jul 24 13:10:19.137582 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:19.137533 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:19.238402 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:19.238324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:20.542944 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:20.542883 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:20.644817 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:20.644766 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:20.748115 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:20.748077 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:20.828950 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:20.828842 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:25.141850 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:25.141810 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:25.238657 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:25.238600 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:25.634009 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:25.633966 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:25.694481 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:25.694439 2558 patch_prober.go:28] interesting pod/image-registry-86db5847b7-cbr7q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:10:25.694691 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:25.694514 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:10:25.704970 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:25.704861 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:27.663843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:27.663809 2558 patch_prober.go:28] interesting pod/image-registry-57bf4c6dfb-zscvm container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:10:27.664359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:27.663865 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" podUID="253c7174-7697-4369-85c6-b0f3abe58c64" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:10:29.742892 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:29.742852 2558 patch_prober.go:28] interesting pod/image-registry-86db5847b7-cbr7q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:10:29.743262 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:29.742914 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:10:30.857196 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:30.857154 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8jsc6" event={"ID":"4971787d-2c92-40a4-94d9-6ab28883e02b","Type":"ContainerStarted","Data":"512cc9a83f4cfdcd7b4838e6d0fc28f6dab0094924c42e891d095579d8c45dc5"} Jul 24 13:10:30.873626 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:30.873558 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8jsc6" podStartSLOduration=74.505667584 podStartE2EDuration="1m32.873535177s" totalImagesPullingTime="18.367867593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:30.87262334 +0000 UTC m=+94.236547433" watchObservedRunningTime="2026-07-24 13:10:30.873535177 +0000 UTC m=+94.237459269" Jul 24 13:10:31.599155 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.599112 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:31.679267 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.679181 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:31.764028 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.763990 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:31.867711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.867671 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" event={"ID":"0825d55b-6343-411d-8a8c-9dcc1d46bf93","Type":"ContainerStarted","Data":"6e7e158be3a7bd6ac98911a1f5b5646957217e741ca12197e853d785233d9c12"} Jul 24 13:10:31.871636 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.871580 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" event={"ID":"e7956822-ac01-4217-89c1-dfb2fdc18fad","Type":"ContainerStarted","Data":"1d235fbebe2cfcc983a053caf2fe9035ef3f0b65486f69eef1afb3d4e16b1540"} Jul 24 13:10:31.873174 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.873117 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:31.877441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.875859 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z9vkb" event={"ID":"b5db3487-f9e9-412c-a5b3-d4e661de0280","Type":"ContainerStarted","Data":"ccce7ebf7cdb4a9c7d38cf680db42006489e8d5c787cf0e1894a4751d7d74bfc"} Jul 24 13:10:31.877441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.875913 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z9vkb" event={"ID":"b5db3487-f9e9-412c-a5b3-d4e661de0280","Type":"ContainerStarted","Data":"bcbb062e46b73e5a9790c8adedda0994ebabca6ce981b75ce6d516ee04694334"} Jul 24 13:10:31.877441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.876515 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z9vkb" Jul 24 13:10:31.877441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.876730 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" Jul 24 13:10:31.879299 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.879186 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56dc49f854-7xh74" event={"ID":"b665f366-5864-4d11-90bd-5020a27f66ab","Type":"ContainerStarted","Data":"c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953"} Jul 24 13:10:31.882905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.882217 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" event={"ID":"9b3afa1c-c462-4ce1-891c-7412da5b4566","Type":"ContainerStarted","Data":"b4ab89ba374c09d9757e277f71683e7c4595a30893a0490cd44e05bda7c92f7f"} Jul 24 13:10:31.886236 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.886118 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" event={"ID":"e983a1fa-d21b-41fc-933d-7ab33bdb5064","Type":"ContainerStarted","Data":"e0d9acf3af154a935b5486cf729b6f75ff3af47c44c88888857626e7e6966fdc"} Jul 24 13:10:31.887818 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.887751 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-5vfs6" podStartSLOduration=41.9049559 podStartE2EDuration="1m3.88773159s" totalImagesPullingTime="21.98277569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:31.887380556 +0000 UTC m=+95.251304647" watchObservedRunningTime="2026-07-24 13:10:31.88773159 +0000 UTC m=+95.251655682" Jul 24 13:10:31.892832 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.892229 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjt56" event={"ID":"aa847532-3ffa-44e0-b42a-0968ac99aade","Type":"ContainerStarted","Data":"11a4dc21f5a78b15d1b6cb10bfd2c054249498f141e1657937259021f080ec65"} Jul 24 13:10:31.896972 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.896944 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" event={"ID":"72e8351b-ad82-4642-b460-974e4f10793f","Type":"ContainerStarted","Data":"da24ba46f24697cc2ab0c34fdf2905ac6ca913ae3d2201008af915f53d649266"} Jul 24 13:10:31.897181 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.897163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" event={"ID":"72e8351b-ad82-4642-b460-974e4f10793f","Type":"ContainerStarted","Data":"148c81c3953b90a33736b906cff0167e04d28b8632cfb54985d1c1a4dc3100d7"} Jul 24 13:10:31.897329 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.897313 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" event={"ID":"72e8351b-ad82-4642-b460-974e4f10793f","Type":"ContainerStarted","Data":"9eee396f9ee80e52a7327e42b561d6fdaa775cc75046eff1ac4eb20eace7a6b3"} Jul 24 13:10:31.902296 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.902268 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-fv88n" event={"ID":"731228c1-003d-4c8b-9f37-2caab3adf288","Type":"ContainerStarted","Data":"fe08a7317ea6ad8c1ddc74bf2fd03155e13b02ceeead4307fc7611a677f0c2fe"} Jul 24 13:10:31.902434 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.902326 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:31.911408 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.911368 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-fv88n" Jul 24 13:10:31.913633 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.913371 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6ffd6fff-mz9fx" podStartSLOduration=3.592112 podStartE2EDuration="25.913350659s" totalImagesPullingTime="22.321238659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:31.909831747 +0000 UTC m=+95.273755839" watchObservedRunningTime="2026-07-24 13:10:31.913350659 +0000 UTC m=+95.277274751" Jul 24 13:10:31.931090 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.930992 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z9vkb" podStartSLOduration=36.990755695 podStartE2EDuration="58.93097151s" totalImagesPullingTime="21.940215815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:31.928702177 +0000 UTC m=+95.292626295" watchObservedRunningTime="2026-07-24 13:10:31.93097151 +0000 UTC m=+95.294895604" Jul 24 13:10:31.933113 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.933061 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:31.959233 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:31.959009 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56dc49f854-7xh74" podStartSLOduration=8.914824823 podStartE2EDuration="21.958984387s" totalImagesPullingTime="13.044159564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:31.95769874 +0000 UTC m=+95.321622860" watchObservedRunningTime="2026-07-24 13:10:31.958984387 +0000 UTC m=+95.322908481" Jul 24 13:10:32.000402 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.000319 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-sbzh4" podStartSLOduration=76.209345919 podStartE2EDuration="1m31.000294653s" totalImagesPullingTime="14.790948734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:31.99502441 +0000 UTC m=+95.358948502" watchObservedRunningTime="2026-07-24 13:10:32.000294653 +0000 UTC m=+95.364218746" Jul 24 13:10:32.056134 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.056043 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fcfb9fbd8-fd24r" podStartSLOduration=3.987890781 podStartE2EDuration="26.0560075s" totalImagesPullingTime="22.068116719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:32.02897816 +0000 UTC m=+95.392902254" watchObservedRunningTime="2026-07-24 13:10:32.0560075 +0000 UTC m=+95.419931589" Jul 24 13:10:32.061031 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.058498 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78cdd7587f-ptkp6" podStartSLOduration=3.945994091 podStartE2EDuration="26.058472306s" totalImagesPullingTime="22.112478215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:32.05121836 +0000 UTC m=+95.415142452" watchObservedRunningTime="2026-07-24 13:10:32.058472306 +0000 UTC m=+95.422396389" Jul 24 13:10:32.061031 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.059612 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:32.081073 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.081012 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-fv88n" podStartSLOduration=2.63005727 podStartE2EDuration="26.080996448s" totalImagesPullingTime="23.450939178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:32.078845582 +0000 UTC m=+95.442769696" watchObservedRunningTime="2026-07-24 13:10:32.080996448 +0000 UTC m=+95.444920539" Jul 24 13:10:32.123598 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.123542 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:32.181364 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.181279 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:32.685050 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.684802 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" podUID="253c7174-7697-4369-85c6-b0f3abe58c64" containerName="registry" containerID="cri-o://7aea9b43cf089b618a0fc813ce68a981d31332f6d98859f18053898e33b9f630" gracePeriod=30 Jul 24 13:10:32.916354 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.916034 2558 generic.go:356] "Generic (PLEG): container finished" podID="253c7174-7697-4369-85c6-b0f3abe58c64" containerID="7aea9b43cf089b618a0fc813ce68a981d31332f6d98859f18053898e33b9f630" exitCode=0 Jul 24 13:10:32.917948 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:32.917047 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" event={"ID":"253c7174-7697-4369-85c6-b0f3abe58c64","Type":"ContainerDied","Data":"7aea9b43cf089b618a0fc813ce68a981d31332f6d98859f18053898e33b9f630"} Jul 24 13:10:33.007196 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.007150 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:10:33.139357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139006 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-bound-sa-token\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139060 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-trusted-ca\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139283 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/253c7174-7697-4369-85c6-b0f3abe58c64-ca-trust-extracted\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139334 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-image-registry-private-configuration\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139362 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-registry-certificates\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139774 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139446 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-installation-pull-secrets\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139774 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139505 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kr6w\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-kube-api-access-5kr6w\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.139774 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.139545 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") pod \"253c7174-7697-4369-85c6-b0f3abe58c64\" (UID: \"253c7174-7697-4369-85c6-b0f3abe58c64\") " Jul 24 13:10:33.141521 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.141458 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:10:33.141720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.141689 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:10:33.143612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.143572 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:10:33.148950 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.148894 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:10:33.152609 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.152359 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/253c7174-7697-4369-85c6-b0f3abe58c64-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:10:33.155271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.155223 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:10:33.155611 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.155579 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-kube-api-access-5kr6w" (OuterVolumeSpecName: "kube-api-access-5kr6w") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "kube-api-access-5kr6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:10:33.157913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.157875 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "253c7174-7697-4369-85c6-b0f3abe58c64" (UID: "253c7174-7697-4369-85c6-b0f3abe58c64"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:10:33.175202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.174305 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86db5847b7-cbr7q"] Jul 24 13:10:33.181922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.181888 2558 patch_prober.go:28] interesting pod/image-registry-86db5847b7-cbr7q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:10:33.182096 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.181964 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240229 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kr6w\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-kube-api-access-5kr6w\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240261 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-registry-tls\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240275 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/253c7174-7697-4369-85c6-b0f3abe58c64-bound-sa-token\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240284 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-trusted-ca\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240292 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/253c7174-7697-4369-85c6-b0f3abe58c64-ca-trust-extracted\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240303 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-image-registry-private-configuration\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240312 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/253c7174-7697-4369-85c6-b0f3abe58c64-registry-certificates\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.240929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.240323 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/253c7174-7697-4369-85c6-b0f3abe58c64-installation-pull-secrets\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:33.241297 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.241267 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:33.338094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.338046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:33.926114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.926082 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" Jul 24 13:10:33.926926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.926826 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57bf4c6dfb-zscvm" event={"ID":"253c7174-7697-4369-85c6-b0f3abe58c64","Type":"ContainerDied","Data":"e701df91b8bc231d3823c016dcd443a4c9fc60a06405ad895c78c56b21d91685"} Jul 24 13:10:33.926926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.926877 2558 scope.go:117] "RemoveContainer" containerID="7aea9b43cf089b618a0fc813ce68a981d31332f6d98859f18053898e33b9f630" Jul 24 13:10:33.945110 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.945072 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57bf4c6dfb-zscvm"] Jul 24 13:10:33.950259 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:33.950219 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57bf4c6dfb-zscvm"] Jul 24 13:10:34.605773 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:34.605737 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:34.642431 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:34.638125 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sdkhm" Jul 24 13:10:34.672472 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:34.672252 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:34.800515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:34.800478 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:34.872825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:34.872740 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:35.016732 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:35.016698 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:35.197912 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:35.197830 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="253c7174-7697-4369-85c6-b0f3abe58c64" path="/var/lib/kubelet/pods/253c7174-7697-4369-85c6-b0f3abe58c64/volumes" Jul 24 13:10:35.244884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:35.244844 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56dc49f854-7xh74"] Jul 24 13:10:35.356156 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:35.356056 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:35.461436 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:35.461317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:35.556429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:35.556321 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:36.943084 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:36.943042 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hjt56" event={"ID":"aa847532-3ffa-44e0-b42a-0968ac99aade","Type":"ContainerStarted","Data":"f0c6dcaec56c2ea9fa6458760afe4a2107e37fa9fb7ce01ab738ebfdbbbc9158"} Jul 24 13:10:36.962465 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:36.962374 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hjt56" podStartSLOduration=19.230778509 podStartE2EDuration="52.962353649s" totalImagesPullingTime="33.73157514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:36.962333616 +0000 UTC m=+100.326257707" watchObservedRunningTime="2026-07-24 13:10:36.962353649 +0000 UTC m=+100.326277742" Jul 24 13:10:37.401125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:37.401080 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:37.749500 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:37.749410 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:37.807914 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:37.807874 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:38.110087 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:38.110046 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:38.160401 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:38.160361 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:38.265414 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:38.265350 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:38.971794 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:38.971650 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.111358 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.111324 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.239156 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.239065 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.267961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.267926 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xx2hc"] Jul 24 13:10:39.268422 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.268371 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="253c7174-7697-4369-85c6-b0f3abe58c64" containerName="registry" Jul 24 13:10:39.268422 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.268408 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="253c7174-7697-4369-85c6-b0f3abe58c64" containerName="registry" Jul 24 13:10:39.268627 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.268501 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="253c7174-7697-4369-85c6-b0f3abe58c64" containerName="registry" Jul 24 13:10:39.293504 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.293470 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.296455 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.296422 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8tk55\"" Jul 24 13:10:39.296837 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.296815 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 13:10:39.297118 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.297099 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 13:10:39.297333 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.297309 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 13:10:39.297537 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.297519 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 13:10:39.298226 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298203 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8880d140-9846-4046-8580-c268d042b180-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298340 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298262 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-sys\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298340 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298336 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-469k7\" (UniqueName: \"kubernetes.io/projected/8880d140-9846-4046-8580-c268d042b180-kube-api-access-469k7\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880d140-9846-4046-8580-c268d042b180-metrics-client-ca\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298450 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-root\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298473 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8880d140-9846-4046-8580-c268d042b180-node-exporter-tls\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298642 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298531 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-node-exporter-wtmp\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298642 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298573 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8880d140-9846-4046-8580-c268d042b180-node-exporter-textfile\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.298642 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.298624 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8880d140-9846-4046-8580-c268d042b180-node-exporter-accelerators-collector-config\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.321232 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.321193 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.390485 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.390445 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.399202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399163 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-469k7\" (UniqueName: \"kubernetes.io/projected/8880d140-9846-4046-8580-c268d042b180-kube-api-access-469k7\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399264 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880d140-9846-4046-8580-c268d042b180-metrics-client-ca\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399300 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-root\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399328 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8880d140-9846-4046-8580-c268d042b180-node-exporter-tls\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399356 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-node-exporter-wtmp\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399417 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8880d140-9846-4046-8580-c268d042b180-node-exporter-textfile\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399442 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8880d140-9846-4046-8580-c268d042b180-node-exporter-accelerators-collector-config\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399487 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8880d140-9846-4046-8580-c268d042b180-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399557 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-sys\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399670 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-sys\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399899 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399723 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-node-exporter-wtmp\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.399899 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.399863 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8880d140-9846-4046-8580-c268d042b180-root\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.413440 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.406742 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8880d140-9846-4046-8580-c268d042b180-node-exporter-accelerators-collector-config\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.413440 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.409075 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8880d140-9846-4046-8580-c268d042b180-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.413440 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.413346 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8880d140-9846-4046-8580-c268d042b180-node-exporter-tls\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.417787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.417659 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8880d140-9846-4046-8580-c268d042b180-node-exporter-textfile\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.418582 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.418533 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-469k7\" (UniqueName: \"kubernetes.io/projected/8880d140-9846-4046-8580-c268d042b180-kube-api-access-469k7\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.418907 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.418890 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8880d140-9846-4046-8580-c268d042b180-metrics-client-ca\") pod \"node-exporter-xx2hc\" (UID: \"8880d140-9846-4046-8580-c268d042b180\") " pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.491825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.491728 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.591706 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.591641 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.609257 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.609220 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xx2hc" Jul 24 13:10:39.646537 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:10:39.646493 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8880d140_9846_4046_8580_c268d042b180.slice/crio-b5bbe08a50aff7b7e6c596bb533a72a70491e0c4ce61b1af73d689e7d50563cd WatchSource:0}: Error finding container b5bbe08a50aff7b7e6c596bb533a72a70491e0c4ce61b1af73d689e7d50563cd: Status 404 returned error can't find the container with id b5bbe08a50aff7b7e6c596bb533a72a70491e0c4ce61b1af73d689e7d50563cd Jul 24 13:10:39.717487 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.717444 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.817797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.817712 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:39.961086 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:39.961034 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xx2hc" event={"ID":"8880d140-9846-4046-8580-c268d042b180","Type":"ContainerStarted","Data":"b5bbe08a50aff7b7e6c596bb533a72a70491e0c4ce61b1af73d689e7d50563cd"} Jul 24 13:10:40.009979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:40.009942 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:40.205109 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:40.205069 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:40.287189 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:40.287148 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:40.411413 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:40.411280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:40.933790 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:40.933756 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:10:41.619218 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:41.619180 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:41.721260 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:41.721227 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:41.821662 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:41.821572 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:41.971094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:41.971051 2558 generic.go:356] "Generic (PLEG): container finished" podID="8880d140-9846-4046-8580-c268d042b180" containerID="c6fc15aabddcbb1e006ad2a9f1bbb5db43500069486b22cca741c6d575b97963" exitCode=0 Jul 24 13:10:41.971347 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:41.971137 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xx2hc" event={"ID":"8880d140-9846-4046-8580-c268d042b180","Type":"ContainerDied","Data":"c6fc15aabddcbb1e006ad2a9f1bbb5db43500069486b22cca741c6d575b97963"} Jul 24 13:10:42.221557 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.221521 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:42.377562 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.377527 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:42.519310 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.519271 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:42.729291 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.729177 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:42.929297 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.929267 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z9vkb" Jul 24 13:10:42.978618 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.978585 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xx2hc" event={"ID":"8880d140-9846-4046-8580-c268d042b180","Type":"ContainerStarted","Data":"0e17743c1995832bac196076bd2c3d962da2e7000112ef3a47d5f32d92d6a73c"} Jul 24 13:10:42.978824 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:42.978647 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xx2hc" event={"ID":"8880d140-9846-4046-8580-c268d042b180","Type":"ContainerStarted","Data":"f4935ea5c34ff0a24cec9a732e533c3444579402bfba58ec2d0aed4dbb0ed588"} Jul 24 13:10:43.001977 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:43.001862 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xx2hc" podStartSLOduration=2.297324154 podStartE2EDuration="4.001842983s" totalImagesPullingTime="1.604872675s" totalInitContainerRuntime="99.646154ms" isStatefulPod=true podCreationTimestamp="2026-07-24 13:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:43.000261625 +0000 UTC m=+106.364185773" watchObservedRunningTime="2026-07-24 13:10:43.001842983 +0000 UTC m=+106.365767073" Jul 24 13:10:43.095787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:43.095759 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:43.178912 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:43.178882 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:43.194144 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:43.194108 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:43.359556 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:43.359520 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:43.461658 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:43.461613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.008727 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.008686 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.087180 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.087141 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.187581 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.187545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.291113 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.291005 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.530609 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.530575 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.610518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.610475 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:44.718134 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:44.718098 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:45.511664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:45.511626 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:49.818929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:49.818889 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:49.919479 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:49.919444 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:51.181939 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:51.181902 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:51.246828 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:51.246787 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:51.781362 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:51.781326 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:51.881489 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:51.881453 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:54.025538 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:54.025504 2558 generic.go:356] "Generic (PLEG): container finished" podID="13956c89-0f25-43f9-be04-ab203a8de1f9" containerID="5e6496d0bf2ae91f78d47552f91357eb9fbbe119111b2015dcb43ec99fb68e2a" exitCode=0 Jul 24 13:10:54.025941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:54.025585 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" event={"ID":"13956c89-0f25-43f9-be04-ab203a8de1f9","Type":"ContainerDied","Data":"5e6496d0bf2ae91f78d47552f91357eb9fbbe119111b2015dcb43ec99fb68e2a"} Jul 24 13:10:54.025941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:54.025899 2558 scope.go:117] "RemoveContainer" containerID="5e6496d0bf2ae91f78d47552f91357eb9fbbe119111b2015dcb43ec99fb68e2a" Jul 24 13:10:55.031490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:55.031455 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh" event={"ID":"13956c89-0f25-43f9-be04-ab203a8de1f9","Type":"ContainerStarted","Data":"d2b69e5e99f9996d936f827d8b60d0651cc33e62efcbc3b9e456525425e0f6e4"} Jul 24 13:10:55.635862 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:55.635826 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:55.690822 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:55.690782 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:56.276886 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:56.276846 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:56.335426 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:56.335366 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:10:58.204640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.204594 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" containerID="cri-o://5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6" gracePeriod=30 Jul 24 13:10:58.507513 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.507479 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:58.571380 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571343 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c4d641-690d-4043-830f-9b2dfd0dabbb-ca-trust-extracted\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571586 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571426 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571586 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571468 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78h2f\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-kube-api-access-78h2f\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571586 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571506 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-bound-sa-token\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571586 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571530 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-installation-pull-secrets\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571791 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571664 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-trusted-ca\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571791 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571730 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-certificates\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.571791 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.571764 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-image-registry-private-configuration\") pod \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\" (UID: \"d0c4d641-690d-4043-830f-9b2dfd0dabbb\") " Jul 24 13:10:58.572100 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.572061 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:10:58.572501 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.572465 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:10:58.574451 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.574398 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:10:58.574451 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.574407 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:10:58.574607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.574489 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:10:58.574607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.574564 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:10:58.574607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.574600 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-kube-api-access-78h2f" (OuterVolumeSpecName: "kube-api-access-78h2f") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "kube-api-access-78h2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:10:58.585314 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.585274 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0c4d641-690d-4043-830f-9b2dfd0dabbb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d0c4d641-690d-4043-830f-9b2dfd0dabbb" (UID: "d0c4d641-690d-4043-830f-9b2dfd0dabbb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:10:58.673502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673449 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78h2f\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-kube-api-access-78h2f\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673496 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-bound-sa-token\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673508 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-installation-pull-secrets\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673517 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-trusted-ca\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673526 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-certificates\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673808 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673535 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d0c4d641-690d-4043-830f-9b2dfd0dabbb-image-registry-private-configuration\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673808 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673544 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d0c4d641-690d-4043-830f-9b2dfd0dabbb-ca-trust-extracted\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:58.673808 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:58.673553 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d0c4d641-690d-4043-830f-9b2dfd0dabbb-registry-tls\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:10:59.047838 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.047798 2558 generic.go:356] "Generic (PLEG): container finished" podID="46b2036d-7e89-44e0-9b58-dd8a30691fd8" containerID="b46263db2bbb3f10bfad8656117779d3e124deb15c811376269cae3a0112551f" exitCode=0 Jul 24 13:10:59.048022 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.047879 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-69k7c" event={"ID":"46b2036d-7e89-44e0-9b58-dd8a30691fd8","Type":"ContainerDied","Data":"b46263db2bbb3f10bfad8656117779d3e124deb15c811376269cae3a0112551f"} Jul 24 13:10:59.048551 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.048522 2558 scope.go:117] "RemoveContainer" containerID="b46263db2bbb3f10bfad8656117779d3e124deb15c811376269cae3a0112551f" Jul 24 13:10:59.049374 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.049354 2558 generic.go:356] "Generic (PLEG): container finished" podID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerID="5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6" exitCode=0 Jul 24 13:10:59.049451 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.049430 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" Jul 24 13:10:59.049451 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.049415 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" event={"ID":"d0c4d641-690d-4043-830f-9b2dfd0dabbb","Type":"ContainerDied","Data":"5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6"} Jul 24 13:10:59.049530 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.049475 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86db5847b7-cbr7q" event={"ID":"d0c4d641-690d-4043-830f-9b2dfd0dabbb","Type":"ContainerDied","Data":"0dcedcc427c9ac67791a85d839f52f9d84fbfb593e601ab69bb583c425a34992"} Jul 24 13:10:59.049530 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.049502 2558 scope.go:117] "RemoveContainer" containerID="5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6" Jul 24 13:10:59.060967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.060944 2558 scope.go:117] "RemoveContainer" containerID="5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6" Jul 24 13:10:59.061221 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:10:59.061200 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6\": container with ID starting with 5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6 not found: ID does not exist" containerID="5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6" Jul 24 13:10:59.061293 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.061232 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6"} err="failed to get container status \"5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6\": rpc error: code = NotFound desc = could not find container \"5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6\": container with ID starting with 5a2f4e0a2189ece52fada3ea9f75a2b0dfffbd028bde2e34ff4d81752facb2d6 not found: ID does not exist" Jul 24 13:10:59.079552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.078720 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-86db5847b7-cbr7q"] Jul 24 13:10:59.080916 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.080889 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-86db5847b7-cbr7q"] Jul 24 13:10:59.195924 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.195889 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" path="/var/lib/kubelet/pods/d0c4d641-690d-4043-830f-9b2dfd0dabbb/volumes" Jul 24 13:10:59.696413 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.696369 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b84687b8c-kmckn_6e76b0c5-9196-42ef-8515-86b4ec2d1f86/router/0.log" Jul 24 13:10:59.702454 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:10:59.702420 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwrp8_2c393ace-cf46-4319-8dc3-548b7bf80ce0/serve-healthcheck-canary/0.log" Jul 24 13:11:00.056013 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.055929 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-69k7c" event={"ID":"46b2036d-7e89-44e0-9b58-dd8a30691fd8","Type":"ContainerStarted","Data":"b0d74870712ee6cb5efdc4d472f5409948713606b6c0f2aa73facfd2c81d9611"} Jul 24 13:11:00.272439 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.272373 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56dc49f854-7xh74" podUID="b665f366-5864-4d11-90bd-5020a27f66ab" containerName="console" containerID="cri-o://c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953" gracePeriod=15 Jul 24 13:11:00.563690 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.563661 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:11:00.689454 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689415 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-serving-cert\") pod \"b665f366-5864-4d11-90bd-5020a27f66ab\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " Jul 24 13:11:00.689638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689496 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbccl\" (UniqueName: \"kubernetes.io/projected/b665f366-5864-4d11-90bd-5020a27f66ab-kube-api-access-kbccl\") pod \"b665f366-5864-4d11-90bd-5020a27f66ab\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " Jul 24 13:11:00.689638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689527 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-console-config\") pod \"b665f366-5864-4d11-90bd-5020a27f66ab\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " Jul 24 13:11:00.689638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689547 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-oauth-serving-cert\") pod \"b665f366-5864-4d11-90bd-5020a27f66ab\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " Jul 24 13:11:00.689638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689571 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-oauth-config\") pod \"b665f366-5864-4d11-90bd-5020a27f66ab\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " Jul 24 13:11:00.689638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689609 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-service-ca\") pod \"b665f366-5864-4d11-90bd-5020a27f66ab\" (UID: \"b665f366-5864-4d11-90bd-5020a27f66ab\") " Jul 24 13:11:00.689984 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689953 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-console-config" (OuterVolumeSpecName: "console-config") pod "b665f366-5864-4d11-90bd-5020a27f66ab" (UID: "b665f366-5864-4d11-90bd-5020a27f66ab"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:00.690053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.689962 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b665f366-5864-4d11-90bd-5020a27f66ab" (UID: "b665f366-5864-4d11-90bd-5020a27f66ab"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:00.690053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.690039 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-service-ca" (OuterVolumeSpecName: "service-ca") pod "b665f366-5864-4d11-90bd-5020a27f66ab" (UID: "b665f366-5864-4d11-90bd-5020a27f66ab"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:00.691986 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.691960 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b665f366-5864-4d11-90bd-5020a27f66ab-kube-api-access-kbccl" (OuterVolumeSpecName: "kube-api-access-kbccl") pod "b665f366-5864-4d11-90bd-5020a27f66ab" (UID: "b665f366-5864-4d11-90bd-5020a27f66ab"). InnerVolumeSpecName "kube-api-access-kbccl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:11:00.691986 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.691966 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b665f366-5864-4d11-90bd-5020a27f66ab" (UID: "b665f366-5864-4d11-90bd-5020a27f66ab"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:11:00.692176 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.692035 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b665f366-5864-4d11-90bd-5020a27f66ab" (UID: "b665f366-5864-4d11-90bd-5020a27f66ab"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:11:00.790710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.790668 2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-oauth-serving-cert\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:11:00.790710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.790705 2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-oauth-config\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:11:00.790710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.790714 2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-service-ca\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:11:00.791169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.790723 2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b665f366-5864-4d11-90bd-5020a27f66ab-console-serving-cert\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:11:00.791169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.790733 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbccl\" (UniqueName: \"kubernetes.io/projected/b665f366-5864-4d11-90bd-5020a27f66ab-kube-api-access-kbccl\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:11:00.791169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:00.790744 2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b665f366-5864-4d11-90bd-5020a27f66ab-console-config\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:11:01.061908 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.061815 2558 generic.go:356] "Generic (PLEG): container finished" podID="b665f366-5864-4d11-90bd-5020a27f66ab" containerID="c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953" exitCode=0 Jul 24 13:11:01.061908 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.061870 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56dc49f854-7xh74" Jul 24 13:11:01.062095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.061898 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56dc49f854-7xh74" event={"ID":"b665f366-5864-4d11-90bd-5020a27f66ab","Type":"ContainerDied","Data":"c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953"} Jul 24 13:11:01.062095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.061955 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56dc49f854-7xh74" event={"ID":"b665f366-5864-4d11-90bd-5020a27f66ab","Type":"ContainerDied","Data":"6afe0dcc554334121227350510355ef7a2a8f4ddc6eec04449beb7abeadb2821"} Jul 24 13:11:01.062095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.061975 2558 scope.go:117] "RemoveContainer" containerID="c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953" Jul 24 13:11:01.070965 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.070946 2558 scope.go:117] "RemoveContainer" containerID="c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953" Jul 24 13:11:01.071259 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:11:01.071236 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953\": container with ID starting with c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953 not found: ID does not exist" containerID="c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953" Jul 24 13:11:01.071307 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.071271 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953"} err="failed to get container status \"c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953\": rpc error: code = NotFound desc = could not find container \"c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953\": container with ID starting with c9dc7b23deed9d231727a0a4d48f03b8d5bd5f421386ec914dba60792c44e953 not found: ID does not exist" Jul 24 13:11:01.083104 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.083068 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56dc49f854-7xh74"] Jul 24 13:11:01.086185 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.086156 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56dc49f854-7xh74"] Jul 24 13:11:01.195813 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.195773 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b665f366-5864-4d11-90bd-5020a27f66ab" path="/var/lib/kubelet/pods/b665f366-5864-4d11-90bd-5020a27f66ab/volumes" Jul 24 13:11:01.486331 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.486291 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:01.544417 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.544352 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:01.604165 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:01.604131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:03.575399 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:03.575345 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:03.627496 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:03.627462 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:09.100960 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:09.100927 2558 generic.go:356] "Generic (PLEG): container finished" podID="31c6e268-c2a1-49b3-b910-e185014fcef8" containerID="293c8ba1fded0adae9c84328a892a16718e1192035418df8311fd185739f7dd6" exitCode=0 Jul 24 13:11:09.101334 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:09.101001 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" event={"ID":"31c6e268-c2a1-49b3-b910-e185014fcef8","Type":"ContainerDied","Data":"293c8ba1fded0adae9c84328a892a16718e1192035418df8311fd185739f7dd6"} Jul 24 13:11:09.101411 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:09.101374 2558 scope.go:117] "RemoveContainer" containerID="293c8ba1fded0adae9c84328a892a16718e1192035418df8311fd185739f7dd6" Jul 24 13:11:10.107507 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:10.107436 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-k8bnk" event={"ID":"31c6e268-c2a1-49b3-b910-e185014fcef8","Type":"ContainerStarted","Data":"f2324379cb11871a5ced18bbdd9be61e6d1148c7e828b57baaba4f613e1f9951"} Jul 24 13:11:18.550432 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:18.550364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:18.650482 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:18.650432 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:18.751619 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:18.751579 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:55.365952 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:55.365867 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:11:55.424159 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:11:55.424113 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:07.217236 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:07.217192 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:07.271860 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:07.271826 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:10.631508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:10.631470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:10.732628 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:10.732589 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:11.102682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:11.102638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:11.162355 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:11.162319 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:11.227379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:11.227337 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:11.814630 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:11.814592 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:11.869322 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:11.869284 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:21.786825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:21.786786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:21.854823 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:21.854750 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:21.915788 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:21.915745 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:29.819652 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:29.819613 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:50.248283 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:50.248238 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:12:50.349636 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:12:50.349599 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:14.930977 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:14.930935 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:15.802122 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:15.802089 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:17.212448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:17.212412 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:17.589498 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:17.589380 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:20.843455 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:20.843352 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:20.894125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:20.894090 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:22.231202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:22.231163 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:22.346826 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:22.346786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:23.250154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:23.250114 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:23.340405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:23.340352 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:25.156018 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:25.155978 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:25.625509 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:25.625477 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:27.624854 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:27.624818 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:27.725059 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:27.725022 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:31.217575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:31.217534 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:32.346753 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:32.346708 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:33.783585 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:33.783549 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:38.175699 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:38.175663 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:38.272632 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:38.272592 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:40.438278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:40.438229 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:53.978265 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:53.978227 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:54.105306 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:54.105265 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:54.637066 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:54.637018 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:54.961441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:54.961314 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.063019 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.062984 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.116175 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116139 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc"] Jul 24 13:13:55.116500 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116485 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" Jul 24 13:13:55.116548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116502 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" Jul 24 13:13:55.116548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116515 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b665f366-5864-4d11-90bd-5020a27f66ab" containerName="console" Jul 24 13:13:55.116548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116521 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b665f366-5864-4d11-90bd-5020a27f66ab" containerName="console" Jul 24 13:13:55.116676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116569 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b665f366-5864-4d11-90bd-5020a27f66ab" containerName="console" Jul 24 13:13:55.116676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.116578 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0c4d641-690d-4043-830f-9b2dfd0dabbb" containerName="registry" Jul 24 13:13:55.119869 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.119841 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.127468 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.127428 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 13:13:55.127641 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.127550 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 13:13:55.127641 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.127623 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 13:13:55.128088 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.127428 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-84bfr\"" Jul 24 13:13:55.128088 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.127558 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 13:13:55.135545 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.135515 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc"] Jul 24 13:13:55.209708 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.209671 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df9f9a58-f2fc-4692-a046-fa35407b5b8e-manifests\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.209926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.209735 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df9f9a58-f2fc-4692-a046-fa35407b5b8e-tmp\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.209926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.209806 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df9f9a58-f2fc-4692-a046-fa35407b5b8e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.209926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.209859 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df9f9a58-f2fc-4692-a046-fa35407b5b8e-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.209926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.209900 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvzd9\" (UniqueName: \"kubernetes.io/projected/df9f9a58-f2fc-4692-a046-fa35407b5b8e-kube-api-access-qvzd9\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.239647 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.239559 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.310478 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.310432 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df9f9a58-f2fc-4692-a046-fa35407b5b8e-manifests\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.310672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.310489 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df9f9a58-f2fc-4692-a046-fa35407b5b8e-tmp\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.310672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.310515 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df9f9a58-f2fc-4692-a046-fa35407b5b8e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.310672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.310545 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df9f9a58-f2fc-4692-a046-fa35407b5b8e-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.310672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.310564 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvzd9\" (UniqueName: \"kubernetes.io/projected/df9f9a58-f2fc-4692-a046-fa35407b5b8e-kube-api-access-qvzd9\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.310891 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.310859 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df9f9a58-f2fc-4692-a046-fa35407b5b8e-manifests\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.312983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.312954 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df9f9a58-f2fc-4692-a046-fa35407b5b8e-tmp\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.313102 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.313043 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df9f9a58-f2fc-4692-a046-fa35407b5b8e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.313199 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.313180 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df9f9a58-f2fc-4692-a046-fa35407b5b8e-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.319598 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.319567 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvzd9\" (UniqueName: \"kubernetes.io/projected/df9f9a58-f2fc-4692-a046-fa35407b5b8e-kube-api-access-qvzd9\") pod \"opendatahub-operator-controller-manager-dbfd75c57-9vzwc\" (UID: \"df9f9a58-f2fc-4692-a046-fa35407b5b8e\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.340203 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.340165 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.431597 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.431556 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:13:55.445947 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.445907 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.542629 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.542583 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.583186 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.583151 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc"] Jul 24 13:13:55.586627 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:13:55.586600 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9f9a58_f2fc_4692_a046_fa35407b5b8e.slice/crio-de37902716e3b1c63152ab803bc99e23a4c45390bdeb3f3f5fbfc928ec6a4fa4 WatchSource:0}: Error finding container de37902716e3b1c63152ab803bc99e23a4c45390bdeb3f3f5fbfc928ec6a4fa4: Status 404 returned error can't find the container with id de37902716e3b1c63152ab803bc99e23a4c45390bdeb3f3f5fbfc928ec6a4fa4 Jul 24 13:13:55.649070 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.649033 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:55.761515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.761478 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" event={"ID":"df9f9a58-f2fc-4692-a046-fa35407b5b8e","Type":"ContainerStarted","Data":"de37902716e3b1c63152ab803bc99e23a4c45390bdeb3f3f5fbfc928ec6a4fa4"} Jul 24 13:13:55.781116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:55.781027 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:56.966087 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:56.966028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:57.024851 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:57.024746 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:13:57.027853 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:57.027760 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:13:57.043706 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:57.043678 2558 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 13:13:59.615286 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:59.615249 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:13:59.784226 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:13:59.784189 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" event={"ID":"df9f9a58-f2fc-4692-a046-fa35407b5b8e","Type":"ContainerStarted","Data":"36a7a285b7e9290da7b43e3145ae4df258d91242fd883712dcb8a1484c907ea2"} Jul 24 13:14:00.629465 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:00.629420 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:00.736916 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:00.736878 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:00.790308 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:00.790270 2558 generic.go:356] "Generic (PLEG): container finished" podID="df9f9a58-f2fc-4692-a046-fa35407b5b8e" containerID="36a7a285b7e9290da7b43e3145ae4df258d91242fd883712dcb8a1484c907ea2" exitCode=0 Jul 24 13:14:00.790570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:00.790366 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" event={"ID":"df9f9a58-f2fc-4692-a046-fa35407b5b8e","Type":"ContainerDied","Data":"36a7a285b7e9290da7b43e3145ae4df258d91242fd883712dcb8a1484c907ea2"} Jul 24 13:14:00.790967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:00.790951 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:14:01.797097 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:01.797001 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" event={"ID":"df9f9a58-f2fc-4692-a046-fa35407b5b8e","Type":"ContainerStarted","Data":"8e1b9b7b27be132937bb5d60360f6ac8564071d5acc5cddc9fe232679a016d3d"} Jul 24 13:14:01.797525 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:01.797121 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:14:01.821364 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:01.821305 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" podStartSLOduration=1.783388344 podStartE2EDuration="6.821286763s" totalImagesPullingTime="3.967731484s" totalInitContainerRuntime="1.070166935s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:13:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:01.819499966 +0000 UTC m=+305.183424056" watchObservedRunningTime="2026-07-24 13:14:01.821286763 +0000 UTC m=+305.185210856" Jul 24 13:14:06.812168 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:06.812131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:06.919949 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:06.919902 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:12.803912 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:12.803879 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-9vzwc" Jul 24 13:14:14.611532 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:14.611489 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:14.757450 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:14.757411 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:15.285864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.285826 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2"] Jul 24 13:14:15.295709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.295678 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.298951 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.298929 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 13:14:15.299058 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.298930 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 13:14:15.299326 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.299308 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-twt4h\"" Jul 24 13:14:15.310200 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.310170 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2"] Jul 24 13:14:15.332503 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.332472 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:15.371360 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.371317 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8839b621-a1fe-47d9-a70f-3ad9db2099d7-tls-certs\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.371584 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.371377 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8839b621-a1fe-47d9-a70f-3ad9db2099d7-tmp\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.371584 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.371451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f65hx\" (UniqueName: \"kubernetes.io/projected/8839b621-a1fe-47d9-a70f-3ad9db2099d7-kube-api-access-f65hx\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.455518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.455475 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:15.472301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.472264 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f65hx\" (UniqueName: \"kubernetes.io/projected/8839b621-a1fe-47d9-a70f-3ad9db2099d7-kube-api-access-f65hx\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.472490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.472349 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8839b621-a1fe-47d9-a70f-3ad9db2099d7-tls-certs\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.472490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.472432 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8839b621-a1fe-47d9-a70f-3ad9db2099d7-tmp\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.475224 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.475173 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8839b621-a1fe-47d9-a70f-3ad9db2099d7-tmp\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.475507 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.475483 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8839b621-a1fe-47d9-a70f-3ad9db2099d7-tls-certs\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.482225 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.482184 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f65hx\" (UniqueName: \"kubernetes.io/projected/8839b621-a1fe-47d9-a70f-3ad9db2099d7-kube-api-access-f65hx\") pod \"kube-auth-proxy-76dcd6cb6b-kf6p2\" (UID: \"8839b621-a1fe-47d9-a70f-3ad9db2099d7\") " pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.564045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.563957 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-png6s"] Jul 24 13:14:15.566425 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.566406 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:15.569802 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.569780 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 24 13:14:15.569802 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.569793 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wrz9c\"" Jul 24 13:14:15.595953 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.595909 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-png6s"] Jul 24 13:14:15.605518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.605487 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" Jul 24 13:14:15.674046 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.674007 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg7r7\" (UniqueName: \"kubernetes.io/projected/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-kube-api-access-bg7r7\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:15.674663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.674637 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-cert\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:15.691116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.691054 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:15.766723 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.766679 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2"] Jul 24 13:14:15.771570 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:14:15.771524 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8839b621_a1fe_47d9_a70f_3ad9db2099d7.slice/crio-e16683a3482fa4cc0b3e4781561b704cb65411f066944d1801a9250064895408 WatchSource:0}: Error finding container e16683a3482fa4cc0b3e4781561b704cb65411f066944d1801a9250064895408: Status 404 returned error can't find the container with id e16683a3482fa4cc0b3e4781561b704cb65411f066944d1801a9250064895408 Jul 24 13:14:15.775266 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.775233 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg7r7\" (UniqueName: \"kubernetes.io/projected/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-kube-api-access-bg7r7\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:15.775424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.775320 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-cert\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:15.775504 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:15.775484 2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 24 13:14:15.775575 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:15.775564 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-cert podName:8e2ffe89-f3f9-4588-9477-7e5609e5a6b4 nodeName:}" failed. No retries permitted until 2026-07-24 13:14:16.275544907 +0000 UTC m=+319.639468977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-cert") pod "odh-model-controller-7998c67dc4-png6s" (UID: "8e2ffe89-f3f9-4588-9477-7e5609e5a6b4") : secret "odh-model-controller-webhook-cert" not found Jul 24 13:14:15.784098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.784067 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg7r7\" (UniqueName: \"kubernetes.io/projected/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-kube-api-access-bg7r7\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:15.818866 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.818765 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:15.856228 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:15.856183 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" event={"ID":"8839b621-a1fe-47d9-a70f-3ad9db2099d7","Type":"ContainerStarted","Data":"e16683a3482fa4cc0b3e4781561b704cb65411f066944d1801a9250064895408"} Jul 24 13:14:16.122079 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.118819 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:16.280845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.280800 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-cert\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:16.283323 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.283298 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e2ffe89-f3f9-4588-9477-7e5609e5a6b4-cert\") pod \"odh-model-controller-7998c67dc4-png6s\" (UID: \"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4\") " pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:16.421510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.421421 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-75q6g"] Jul 24 13:14:16.423987 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.423962 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:16.426616 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.426592 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 13:14:16.458669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.458619 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:16.476600 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.476557 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:16.481825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.481788 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-75q6g"] Jul 24 13:14:16.482508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.482477 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb8dw\" (UniqueName: \"kubernetes.io/projected/39e86268-e319-40ae-811a-0d3390f5d16e-kube-api-access-cb8dw\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:16.482699 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.482616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e86268-e319-40ae-811a-0d3390f5d16e-tls-certs\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:16.584210 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.583959 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e86268-e319-40ae-811a-0d3390f5d16e-tls-certs\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:16.584210 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.584029 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb8dw\" (UniqueName: \"kubernetes.io/projected/39e86268-e319-40ae-811a-0d3390f5d16e-kube-api-access-cb8dw\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:16.584210 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:16.584114 2558 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 24 13:14:16.584210 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:16.584210 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e86268-e319-40ae-811a-0d3390f5d16e-tls-certs podName:39e86268-e319-40ae-811a-0d3390f5d16e nodeName:}" failed. No retries permitted until 2026-07-24 13:14:17.084186124 +0000 UTC m=+320.448110194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/39e86268-e319-40ae-811a-0d3390f5d16e-tls-certs") pod "model-serving-api-54d89bd5fc-75q6g" (UID: "39e86268-e319-40ae-811a-0d3390f5d16e") : secret "model-serving-api-tls" not found Jul 24 13:14:16.600600 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.600567 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb8dw\" (UniqueName: \"kubernetes.io/projected/39e86268-e319-40ae-811a-0d3390f5d16e-kube-api-access-cb8dw\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:16.673490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.673253 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-png6s"] Jul 24 13:14:16.676858 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:14:16.676796 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2ffe89_f3f9_4588_9477_7e5609e5a6b4.slice/crio-ade1c2a5d097b5ef99d12d765fd16ff413b41dc94527b2eed0766e77de730796 WatchSource:0}: Error finding container ade1c2a5d097b5ef99d12d765fd16ff413b41dc94527b2eed0766e77de730796: Status 404 returned error can't find the container with id ade1c2a5d097b5ef99d12d765fd16ff413b41dc94527b2eed0766e77de730796 Jul 24 13:14:16.773796 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.773759 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:16.782683 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.782640 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g"] Jul 24 13:14:16.785885 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.785845 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.788497 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.788466 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 13:14:16.788651 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.788621 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 13:14:16.788747 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.788732 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-db6c7\"" Jul 24 13:14:16.788828 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.788809 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 13:14:16.789003 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.788986 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 13:14:16.806410 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.806339 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g"] Jul 24 13:14:16.848506 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.848453 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:16.862604 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.862564 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" event={"ID":"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4","Type":"ContainerStarted","Data":"ade1c2a5d097b5ef99d12d765fd16ff413b41dc94527b2eed0766e77de730796"} Jul 24 13:14:16.886363 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886316 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxfxv\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-kube-api-access-lxfxv\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.886596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.886596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886453 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.886596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886497 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.886764 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886585 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173c36e6-0bcd-4c47-9032-4322b2a76406-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.886764 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886626 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.886764 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.886654 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987456 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987352 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987456 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987431 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987497 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173c36e6-0bcd-4c47-9032-4322b2a76406-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987524 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987554 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987609 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxfxv\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-kube-api-access-lxfxv\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.987681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.987665 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.988954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.988923 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.990454 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.990425 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173c36e6-0bcd-4c47-9032-4322b2a76406-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.990933 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.990904 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.991213 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.991188 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.991292 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.991264 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.996336 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.996298 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxfxv\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-kube-api-access-lxfxv\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:16.996525 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:16.996504 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tsx6g\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:17.072341 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.072273 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:17.088121 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.088075 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e86268-e319-40ae-811a-0d3390f5d16e-tls-certs\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:17.090648 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.090621 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39e86268-e319-40ae-811a-0d3390f5d16e-tls-certs\") pod \"model-serving-api-54d89bd5fc-75q6g\" (UID: \"39e86268-e319-40ae-811a-0d3390f5d16e\") " pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:17.099551 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.099514 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:17.269172 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.269139 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g"] Jul 24 13:14:17.272648 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:14:17.272608 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173c36e6_0bcd_4c47_9032_4322b2a76406.slice/crio-9bac4e08700fa064e8cc3e3f57f6ddeaaf35f77f725d0dd0fdbdbefbf0bf079b WatchSource:0}: Error finding container 9bac4e08700fa064e8cc3e3f57f6ddeaaf35f77f725d0dd0fdbdbefbf0bf079b: Status 404 returned error can't find the container with id 9bac4e08700fa064e8cc3e3f57f6ddeaaf35f77f725d0dd0fdbdbefbf0bf079b Jul 24 13:14:17.339076 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.339040 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:17.509462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.509421 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-75q6g"] Jul 24 13:14:17.833145 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.833102 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:17.871715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:17.871673 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" event={"ID":"173c36e6-0bcd-4c47-9032-4322b2a76406","Type":"ContainerStarted","Data":"9bac4e08700fa064e8cc3e3f57f6ddeaaf35f77f725d0dd0fdbdbefbf0bf079b"} Jul 24 13:14:18.289091 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:14:18.289042 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e86268_e319_40ae_811a_0d3390f5d16e.slice/crio-2699e1b735e96b4b46ecfffeff6d20ec7bcb1038285502f4e42179ebc3ff1e69 WatchSource:0}: Error finding container 2699e1b735e96b4b46ecfffeff6d20ec7bcb1038285502f4e42179ebc3ff1e69: Status 404 returned error can't find the container with id 2699e1b735e96b4b46ecfffeff6d20ec7bcb1038285502f4e42179ebc3ff1e69 Jul 24 13:14:18.878760 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:18.878719 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" event={"ID":"39e86268-e319-40ae-811a-0d3390f5d16e","Type":"ContainerStarted","Data":"2699e1b735e96b4b46ecfffeff6d20ec7bcb1038285502f4e42179ebc3ff1e69"} Jul 24 13:14:21.243369 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.243331 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:21.243701 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.243440 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:21.897168 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.897125 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" event={"ID":"8e2ffe89-f3f9-4588-9477-7e5609e5a6b4","Type":"ContainerStarted","Data":"10a8e26bdcb07fc6b28a65a0da159fe8a67dbffe762062a98d26f7fcf6b40696"} Jul 24 13:14:21.897865 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.897839 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:21.900042 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.900008 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" event={"ID":"8839b621-a1fe-47d9-a70f-3ad9db2099d7","Type":"ContainerStarted","Data":"17da117239698afaaba68e4be37529be682fb2e2cc7399436fe4e9ded1a3b73f"} Jul 24 13:14:21.902421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.902369 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" event={"ID":"39e86268-e319-40ae-811a-0d3390f5d16e","Type":"ContainerStarted","Data":"fa45091862a925976e587f578d6c9cd258c81290b75d27d9c67e624cd9e37c86"} Jul 24 13:14:21.902905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.902883 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:14:21.905481 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.905453 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" event={"ID":"173c36e6-0bcd-4c47-9032-4322b2a76406","Type":"ContainerStarted","Data":"ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d"} Jul 24 13:14:21.905985 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.905962 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:21.958690 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.958613 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76dcd6cb6b-kf6p2" podStartSLOduration=4.386393074 podStartE2EDuration="6.958589443s" totalImagesPullingTime="2.572196369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:21.956324018 +0000 UTC m=+325.320248110" watchObservedRunningTime="2026-07-24 13:14:21.958589443 +0000 UTC m=+325.322513535" Jul 24 13:14:21.958962 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.958928 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" podStartSLOduration=2.401531155 podStartE2EDuration="6.958918103s" totalImagesPullingTime="4.557386948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:21.932745367 +0000 UTC m=+325.296669459" watchObservedRunningTime="2026-07-24 13:14:21.958918103 +0000 UTC m=+325.322842263" Jul 24 13:14:21.988288 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.985009 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" podStartSLOduration=2.017087787 podStartE2EDuration="5.984978678s" totalImagesPullingTime="3.967890891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:21.98323289 +0000 UTC m=+325.347156985" watchObservedRunningTime="2026-07-24 13:14:21.984978678 +0000 UTC m=+325.348902770" Jul 24 13:14:21.989014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:21.988971 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:22.007350 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:22.007289 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" podStartSLOduration=3.052603535 podStartE2EDuration="6.00726286s" totalImagesPullingTime="2.954659325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:22.005324097 +0000 UTC m=+325.369248189" watchObservedRunningTime="2026-07-24 13:14:22.00726286 +0000 UTC m=+325.371186951" Jul 24 13:14:22.089084 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:22.089041 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:22.780972 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:22.780936 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:22.912530 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:22.912502 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:22.957462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:22.957425 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:23.026570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:23.026536 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:25.908212 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:25.908178 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:25.996458 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:25.996425 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7"] Jul 24 13:14:26.000038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.000019 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.003842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.003812 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 24 13:14:26.003993 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.003892 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zc268\"" Jul 24 13:14:26.015290 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.015256 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7"] Jul 24 13:14:26.017291 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.017260 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:26.027571 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.027533 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd"] Jul 24 13:14:26.031063 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.031037 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.034374 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.034348 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jsvrc\"" Jul 24 13:14:26.034697 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.034682 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 24 13:14:26.044780 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.044675 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd"] Jul 24 13:14:26.079125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.079078 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-cert\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.079310 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.079140 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzzx4\" (UniqueName: \"kubernetes.io/projected/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-kube-api-access-jzzx4\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.180568 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.180471 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-cert\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.180568 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.180532 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdqw\" (UniqueName: \"kubernetes.io/projected/aac51a14-c7af-49fe-8b35-f31eb4d4de05-kube-api-access-dwdqw\") pod \"llmisvc-controller-manager-777c6bd7d5-nh9xd\" (UID: \"aac51a14-c7af-49fe-8b35-f31eb4d4de05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.180811 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.180588 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzzx4\" (UniqueName: \"kubernetes.io/projected/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-kube-api-access-jzzx4\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.180811 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.180617 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aac51a14-c7af-49fe-8b35-f31eb4d4de05-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nh9xd\" (UID: \"aac51a14-c7af-49fe-8b35-f31eb4d4de05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.180811 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:26.180653 2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 24 13:14:26.180811 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:26.180739 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-cert podName:b805c8df-6b44-456f-bddd-7e4aa2ac49f7 nodeName:}" failed. No retries permitted until 2026-07-24 13:14:26.680722217 +0000 UTC m=+330.044646289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-cert") pod "kserve-controller-manager-d6f4b65b8-9gwb7" (UID: "b805c8df-6b44-456f-bddd-7e4aa2ac49f7") : secret "kserve-webhook-server-cert" not found Jul 24 13:14:26.203775 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.203722 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzzx4\" (UniqueName: \"kubernetes.io/projected/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-kube-api-access-jzzx4\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.281556 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.281509 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdqw\" (UniqueName: \"kubernetes.io/projected/aac51a14-c7af-49fe-8b35-f31eb4d4de05-kube-api-access-dwdqw\") pod \"llmisvc-controller-manager-777c6bd7d5-nh9xd\" (UID: \"aac51a14-c7af-49fe-8b35-f31eb4d4de05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.281720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.281613 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aac51a14-c7af-49fe-8b35-f31eb4d4de05-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nh9xd\" (UID: \"aac51a14-c7af-49fe-8b35-f31eb4d4de05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.284205 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.284175 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aac51a14-c7af-49fe-8b35-f31eb4d4de05-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nh9xd\" (UID: \"aac51a14-c7af-49fe-8b35-f31eb4d4de05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.291820 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.291789 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdqw\" (UniqueName: \"kubernetes.io/projected/aac51a14-c7af-49fe-8b35-f31eb4d4de05-kube-api-access-dwdqw\") pod \"llmisvc-controller-manager-777c6bd7d5-nh9xd\" (UID: \"aac51a14-c7af-49fe-8b35-f31eb4d4de05\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.344658 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.344613 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:26.365845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.365809 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:26.482811 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.482728 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:26.516234 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.516199 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd"] Jul 24 13:14:26.517266 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:14:26.517243 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaac51a14_c7af_49fe_8b35_f31eb4d4de05.slice/crio-ad8cd5058b70d692dec0ee5ec9b8b10bfd361e8f35376b3cc1bafb44f9562c1a WatchSource:0}: Error finding container ad8cd5058b70d692dec0ee5ec9b8b10bfd361e8f35376b3cc1bafb44f9562c1a: Status 404 returned error can't find the container with id ad8cd5058b70d692dec0ee5ec9b8b10bfd361e8f35376b3cc1bafb44f9562c1a Jul 24 13:14:26.684004 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.683964 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-cert\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.686453 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.686430 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b805c8df-6b44-456f-bddd-7e4aa2ac49f7-cert\") pod \"kserve-controller-manager-d6f4b65b8-9gwb7\" (UID: \"b805c8df-6b44-456f-bddd-7e4aa2ac49f7\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.911057 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.911022 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:26.931661 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:26.931622 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" event={"ID":"aac51a14-c7af-49fe-8b35-f31eb4d4de05","Type":"ContainerStarted","Data":"ad8cd5058b70d692dec0ee5ec9b8b10bfd361e8f35376b3cc1bafb44f9562c1a"} Jul 24 13:14:27.073404 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:27.073358 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7"] Jul 24 13:14:27.076117 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:14:27.076089 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb805c8df_6b44_456f_bddd_7e4aa2ac49f7.slice/crio-0921df1be44dcd462f61770686d382a0776ec3f0ae796a52f73a03106686fb7a WatchSource:0}: Error finding container 0921df1be44dcd462f61770686d382a0776ec3f0ae796a52f73a03106686fb7a: Status 404 returned error can't find the container with id 0921df1be44dcd462f61770686d382a0776ec3f0ae796a52f73a03106686fb7a Jul 24 13:14:27.212975 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:27.212896 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:27.310072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:27.310035 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:27.426608 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:27.426536 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:14:27.940021 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:27.939982 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" event={"ID":"b805c8df-6b44-456f-bddd-7e4aa2ac49f7","Type":"ContainerStarted","Data":"0921df1be44dcd462f61770686d382a0776ec3f0ae796a52f73a03106686fb7a"} Jul 24 13:14:28.274692 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.274513 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g"] Jul 24 13:14:28.274870 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.274762 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" podUID="173c36e6-0bcd-4c47-9032-4322b2a76406" containerName="discovery" containerID="cri-o://ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d" gracePeriod=30 Jul 24 13:14:28.544969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.544941 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:28.705941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.705891 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-kubeconfig\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.706136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.705949 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-dns-cert\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.706136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.706033 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxfxv\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-kube-api-access-lxfxv\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.706263 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.706167 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-token\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.706263 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.706210 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173c36e6-0bcd-4c47-9032-4322b2a76406-local-certs\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.706364 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.706264 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-ca-configmap\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.706364 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.706300 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-cacerts\") pod \"173c36e6-0bcd-4c47-9032-4322b2a76406\" (UID: \"173c36e6-0bcd-4c47-9032-4322b2a76406\") " Jul 24 13:14:28.707670 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.707636 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:14:28.709854 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.709806 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:14:28.709982 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.709943 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-kube-api-access-lxfxv" (OuterVolumeSpecName: "kube-api-access-lxfxv") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "kube-api-access-lxfxv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:14:28.710676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.710640 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:14:28.710954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.710929 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/173c36e6-0bcd-4c47-9032-4322b2a76406-local-certs" (OuterVolumeSpecName: "local-certs") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:14:28.711293 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.711267 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-cacerts" (OuterVolumeSpecName: "cacerts") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:14:28.711929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.711901 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-token" (OuterVolumeSpecName: "istio-token") pod "173c36e6-0bcd-4c47-9032-4322b2a76406" (UID: "173c36e6-0bcd-4c47-9032-4322b2a76406"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:14:28.807419 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807313 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxfxv\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-kube-api-access-lxfxv\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.807419 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807352 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-token\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.807419 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807369 2558 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/173c36e6-0bcd-4c47-9032-4322b2a76406-local-certs\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.807419 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807397 2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-ca-configmap\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.807419 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807414 2558 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-cacerts\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.807793 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807429 2558 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-kubeconfig\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.807793 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.807443 2558 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/173c36e6-0bcd-4c47-9032-4322b2a76406-istio-csr-dns-cert\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:14:28.948576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.948527 2558 generic.go:356] "Generic (PLEG): container finished" podID="173c36e6-0bcd-4c47-9032-4322b2a76406" containerID="ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d" exitCode=0 Jul 24 13:14:28.949075 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.948579 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" Jul 24 13:14:28.949075 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.948616 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" event={"ID":"173c36e6-0bcd-4c47-9032-4322b2a76406","Type":"ContainerDied","Data":"ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d"} Jul 24 13:14:28.949075 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.948682 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g" event={"ID":"173c36e6-0bcd-4c47-9032-4322b2a76406","Type":"ContainerDied","Data":"9bac4e08700fa064e8cc3e3f57f6ddeaaf35f77f725d0dd0fdbdbefbf0bf079b"} Jul 24 13:14:28.949075 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.948717 2558 scope.go:117] "RemoveContainer" containerID="ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d" Jul 24 13:14:28.976257 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.976221 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g"] Jul 24 13:14:28.980964 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:28.980933 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tsx6g"] Jul 24 13:14:29.197647 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:29.197609 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="173c36e6-0bcd-4c47-9032-4322b2a76406" path="/var/lib/kubelet/pods/173c36e6-0bcd-4c47-9032-4322b2a76406/volumes" Jul 24 13:14:29.992362 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:29.992226 2558 scope.go:117] "RemoveContainer" containerID="ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d" Jul 24 13:14:29.992690 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:14:29.992668 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d\": container with ID starting with ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d not found: ID does not exist" containerID="ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d" Jul 24 13:14:29.992753 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:29.992709 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d"} err="failed to get container status \"ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d\": rpc error: code = NotFound desc = could not find container \"ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d\": container with ID starting with ef37abb3c179bf3ef2950fe16dc748fc785aa4cce7f5d516142e938e4358569d not found: ID does not exist" Jul 24 13:14:30.961029 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:30.960977 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" event={"ID":"aac51a14-c7af-49fe-8b35-f31eb4d4de05","Type":"ContainerStarted","Data":"981453addb8b9f4b738620cb57c46c7ef9b46d4b18abf0be465cc6871a8727c7"} Jul 24 13:14:30.961238 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:30.961113 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:14:30.962636 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:30.962614 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" event={"ID":"b805c8df-6b44-456f-bddd-7e4aa2ac49f7","Type":"ContainerStarted","Data":"af9846513cbf2372aa8690bbc2cf9a5b2a66403894e333af5c5eee9c05c578d4"} Jul 24 13:14:30.962747 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:30.962665 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:14:30.978627 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:30.978577 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" podStartSLOduration=2.491462406 podStartE2EDuration="5.978561891s" totalImagesPullingTime="3.487099485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:30.977531214 +0000 UTC m=+334.341455319" watchObservedRunningTime="2026-07-24 13:14:30.978561891 +0000 UTC m=+334.342485981" Jul 24 13:14:30.994808 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:30.994745 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" podStartSLOduration=3.069996758 podStartE2EDuration="5.994725135s" totalImagesPullingTime="2.924728377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:30.993300659 +0000 UTC m=+334.357224748" watchObservedRunningTime="2026-07-24 13:14:30.994725135 +0000 UTC m=+334.358649227" Jul 24 13:14:33.917010 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:33.916979 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-png6s" Jul 24 13:14:33.918791 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:14:33.918768 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-75q6g" Jul 24 13:15:01.970665 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:01.970631 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nh9xd" Jul 24 13:15:01.973837 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:01.973812 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9gwb7" Jul 24 13:15:16.149886 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.149849 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:16.252129 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.252085 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:16.413066 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.412968 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gdf87"] Jul 24 13:15:16.413555 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.413530 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="173c36e6-0bcd-4c47-9032-4322b2a76406" containerName="discovery" Jul 24 13:15:16.413555 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.413557 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="173c36e6-0bcd-4c47-9032-4322b2a76406" containerName="discovery" Jul 24 13:15:16.413739 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.413677 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="173c36e6-0bcd-4c47-9032-4322b2a76406" containerName="discovery" Jul 24 13:15:16.425560 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.425529 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:16.428625 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.428589 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:15:16.428974 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.428951 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-xjspb\"" Jul 24 13:15:16.429449 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.429427 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:15:16.429896 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.429847 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gdf87"] Jul 24 13:15:16.470109 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.470075 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:16.510983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.510940 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfd7g\" (UniqueName: \"kubernetes.io/projected/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7-kube-api-access-gfd7g\") pod \"kuadrant-operator-catalog-gdf87\" (UID: \"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7\") " pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:16.611659 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.611622 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfd7g\" (UniqueName: \"kubernetes.io/projected/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7-kube-api-access-gfd7g\") pod \"kuadrant-operator-catalog-gdf87\" (UID: \"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7\") " pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:16.621123 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.621087 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfd7g\" (UniqueName: \"kubernetes.io/projected/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7-kube-api-access-gfd7g\") pod \"kuadrant-operator-catalog-gdf87\" (UID: \"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7\") " pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:16.661400 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.661361 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gdf87"] Jul 24 13:15:16.661662 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.661649 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:16.801442 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.801414 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gdf87"] Jul 24 13:15:16.803109 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:15:16.803084 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe9ca8a3_e7c9_4658_8623_1d8e57cd6af7.slice/crio-933c1f1cdda2efcf9da3ce3dcb510fb3c8c0e8fe5f502f50c808dd1bfd1f3b1b WatchSource:0}: Error finding container 933c1f1cdda2efcf9da3ce3dcb510fb3c8c0e8fe5f502f50c808dd1bfd1f3b1b: Status 404 returned error can't find the container with id 933c1f1cdda2efcf9da3ce3dcb510fb3c8c0e8fe5f502f50c808dd1bfd1f3b1b Jul 24 13:15:16.864766 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.864723 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xwn24"] Jul 24 13:15:16.869207 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.869183 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:16.876002 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.875973 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xwn24"] Jul 24 13:15:16.913700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:16.913645 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4vjw\" (UniqueName: \"kubernetes.io/projected/5ad7204c-6003-4ce5-9d4a-395470a048fb-kube-api-access-m4vjw\") pod \"kuadrant-operator-catalog-xwn24\" (UID: \"5ad7204c-6003-4ce5-9d4a-395470a048fb\") " pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:17.015170 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.015064 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4vjw\" (UniqueName: \"kubernetes.io/projected/5ad7204c-6003-4ce5-9d4a-395470a048fb-kube-api-access-m4vjw\") pod \"kuadrant-operator-catalog-xwn24\" (UID: \"5ad7204c-6003-4ce5-9d4a-395470a048fb\") " pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:17.024682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.024654 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4vjw\" (UniqueName: \"kubernetes.io/projected/5ad7204c-6003-4ce5-9d4a-395470a048fb-kube-api-access-m4vjw\") pod \"kuadrant-operator-catalog-xwn24\" (UID: \"5ad7204c-6003-4ce5-9d4a-395470a048fb\") " pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:17.108585 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.108553 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:17.179961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.179932 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:17.181247 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.181213 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" event={"ID":"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7","Type":"ContainerStarted","Data":"933c1f1cdda2efcf9da3ce3dcb510fb3c8c0e8fe5f502f50c808dd1bfd1f3b1b"} Jul 24 13:15:17.290148 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.290083 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:17.342577 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:17.342554 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xwn24"] Jul 24 13:15:17.345195 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:15:17.345169 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad7204c_6003_4ce5_9d4a_395470a048fb.slice/crio-a548640aca65c4595593699204e1339516aca7643ffe20b82f5c6dc3984aa594 WatchSource:0}: Error finding container a548640aca65c4595593699204e1339516aca7643ffe20b82f5c6dc3984aa594: Status 404 returned error can't find the container with id a548640aca65c4595593699204e1339516aca7643ffe20b82f5c6dc3984aa594 Jul 24 13:15:18.190309 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:18.190272 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" event={"ID":"5ad7204c-6003-4ce5-9d4a-395470a048fb","Type":"ContainerStarted","Data":"a548640aca65c4595593699204e1339516aca7643ffe20b82f5c6dc3984aa594"} Jul 24 13:15:19.197245 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.197150 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" event={"ID":"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7","Type":"ContainerStarted","Data":"c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19"} Jul 24 13:15:19.197245 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.197218 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" podUID="be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" containerName="registry-server" containerID="cri-o://c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19" gracePeriod=2 Jul 24 13:15:19.198798 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.198774 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" event={"ID":"5ad7204c-6003-4ce5-9d4a-395470a048fb","Type":"ContainerStarted","Data":"5e74f0c8e78014b343299c655d0fa17703ccfc464c4b7fb2e9d7a136ce6bca5f"} Jul 24 13:15:19.213597 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.213546 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" podStartSLOduration=1.149224263 podStartE2EDuration="3.213528238s" totalImagesPullingTime="2.064303975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:15:19.21223017 +0000 UTC m=+382.576154262" watchObservedRunningTime="2026-07-24 13:15:19.213528238 +0000 UTC m=+382.577452329" Jul 24 13:15:19.228568 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.228522 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" podStartSLOduration=1.705341494 podStartE2EDuration="3.228506266s" totalImagesPullingTime="1.523164772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:15:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:15:19.22699914 +0000 UTC m=+382.590923230" watchObservedRunningTime="2026-07-24 13:15:19.228506266 +0000 UTC m=+382.592430357" Jul 24 13:15:19.449139 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.449085 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:19.540637 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.540600 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfd7g\" (UniqueName: \"kubernetes.io/projected/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7-kube-api-access-gfd7g\") pod \"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7\" (UID: \"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7\") " Jul 24 13:15:19.542931 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.542898 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7-kube-api-access-gfd7g" (OuterVolumeSpecName: "kube-api-access-gfd7g") pod "be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" (UID: "be9ca8a3-e7c9-4658-8623-1d8e57cd6af7"). InnerVolumeSpecName "kube-api-access-gfd7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:15:19.642214 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:19.642168 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfd7g\" (UniqueName: \"kubernetes.io/projected/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7-kube-api-access-gfd7g\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:15:20.205343 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.205249 2558 generic.go:356] "Generic (PLEG): container finished" podID="be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" containerID="c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19" exitCode=0 Jul 24 13:15:20.205343 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.205292 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" Jul 24 13:15:20.205840 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.205338 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" event={"ID":"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7","Type":"ContainerDied","Data":"c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19"} Jul 24 13:15:20.205840 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.205403 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gdf87" event={"ID":"be9ca8a3-e7c9-4658-8623-1d8e57cd6af7","Type":"ContainerDied","Data":"933c1f1cdda2efcf9da3ce3dcb510fb3c8c0e8fe5f502f50c808dd1bfd1f3b1b"} Jul 24 13:15:20.205840 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.205426 2558 scope.go:117] "RemoveContainer" containerID="c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19" Jul 24 13:15:20.215669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.215649 2558 scope.go:117] "RemoveContainer" containerID="c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19" Jul 24 13:15:20.215951 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:15:20.215930 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19\": container with ID starting with c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19 not found: ID does not exist" containerID="c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19" Jul 24 13:15:20.216002 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.215962 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19"} err="failed to get container status \"c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19\": rpc error: code = NotFound desc = could not find container \"c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19\": container with ID starting with c656a901d53eb827caa647532722fa88556bcec9331e9fd7ee6f67864af2ad19 not found: ID does not exist" Jul 24 13:15:20.230913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.230886 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gdf87"] Jul 24 13:15:20.235694 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:20.235671 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gdf87"] Jul 24 13:15:21.196727 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:21.196691 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" path="/var/lib/kubelet/pods/be9ca8a3-e7c9-4658-8623-1d8e57cd6af7/volumes" Jul 24 13:15:24.071237 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:24.071191 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:24.160278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:24.160242 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:27.180055 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:27.180014 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:27.180523 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:27.180085 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:27.203723 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:27.203695 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:27.257484 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:27.257451 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xwn24" Jul 24 13:15:42.680992 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:42.680948 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:46.463297 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:46.463250 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:48.151293 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.151255 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:48.241514 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.241476 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j"] Jul 24 13:15:48.241856 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.241843 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" containerName="registry-server" Jul 24 13:15:48.241906 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.241857 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" containerName="registry-server" Jul 24 13:15:48.241944 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.241927 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="be9ca8a3-e7c9-4658-8623-1d8e57cd6af7" containerName="registry-server" Jul 24 13:15:48.248527 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.248498 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.251360 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.251331 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mrhqc\"" Jul 24 13:15:48.257725 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.257696 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j"] Jul 24 13:15:48.292804 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.292765 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dzpz\" (UniqueName: \"kubernetes.io/projected/16508327-9ed0-43db-bca7-75393fd4359c-kube-api-access-8dzpz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.293026 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.292906 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16508327-9ed0-43db-bca7-75393fd4359c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.393864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.393823 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dzpz\" (UniqueName: \"kubernetes.io/projected/16508327-9ed0-43db-bca7-75393fd4359c-kube-api-access-8dzpz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.393864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.393872 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16508327-9ed0-43db-bca7-75393fd4359c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.394235 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.394215 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16508327-9ed0-43db-bca7-75393fd4359c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.407416 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.407311 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dzpz\" (UniqueName: \"kubernetes.io/projected/16508327-9ed0-43db-bca7-75393fd4359c-kube-api-access-8dzpz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.560002 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.559960 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:48.712757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.712673 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:48.729999 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:48.729974 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j"] Jul 24 13:15:48.732563 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:15:48.732529 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16508327_9ed0_43db_bca7_75393fd4359c.slice/crio-181705bd21c4b0085962574915ff1d7be719289cec32acf2f52c1bd34d06b95d WatchSource:0}: Error finding container 181705bd21c4b0085962574915ff1d7be719289cec32acf2f52c1bd34d06b95d: Status 404 returned error can't find the container with id 181705bd21c4b0085962574915ff1d7be719289cec32acf2f52c1bd34d06b95d Jul 24 13:15:49.351225 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:49.351189 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" event={"ID":"16508327-9ed0-43db-bca7-75393fd4359c","Type":"ContainerStarted","Data":"181705bd21c4b0085962574915ff1d7be719289cec32acf2f52c1bd34d06b95d"} Jul 24 13:15:50.190735 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:50.190696 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:50.310281 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:50.310246 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:51.293365 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.293327 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb"] Jul 24 13:15:51.296750 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.296726 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:15:51.299379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.299354 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8wmhh\"" Jul 24 13:15:51.320483 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.320451 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb"] Jul 24 13:15:51.322003 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.321954 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfw8b\" (UniqueName: \"kubernetes.io/projected/5a6b3ff9-4e43-47e0-b518-d7f71fe75e30-kube-api-access-vfw8b\") pod \"authorino-operator-75cc8d5cb7-xbgcb\" (UID: \"5a6b3ff9-4e43-47e0-b518-d7f71fe75e30\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:15:51.422785 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.422732 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfw8b\" (UniqueName: \"kubernetes.io/projected/5a6b3ff9-4e43-47e0-b518-d7f71fe75e30-kube-api-access-vfw8b\") pod \"authorino-operator-75cc8d5cb7-xbgcb\" (UID: \"5a6b3ff9-4e43-47e0-b518-d7f71fe75e30\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:15:51.440901 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.440870 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfw8b\" (UniqueName: \"kubernetes.io/projected/5a6b3ff9-4e43-47e0-b518-d7f71fe75e30-kube-api-access-vfw8b\") pod \"authorino-operator-75cc8d5cb7-xbgcb\" (UID: \"5a6b3ff9-4e43-47e0-b518-d7f71fe75e30\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:15:51.611094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.611051 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:15:51.733841 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.733785 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:51.828283 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:51.828223 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb"] Jul 24 13:15:52.041198 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:52.041115 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:52.971349 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:15:52.971308 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6b3ff9_4e43_47e0_b518_d7f71fe75e30.slice/crio-0c3ff9e419431711000be5c6ebbdd9b061cf747d8809f73a1b15efa8413c58c2 WatchSource:0}: Error finding container 0c3ff9e419431711000be5c6ebbdd9b061cf747d8809f73a1b15efa8413c58c2: Status 404 returned error can't find the container with id 0c3ff9e419431711000be5c6ebbdd9b061cf747d8809f73a1b15efa8413c58c2 Jul 24 13:15:53.041856 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.041821 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cd96db8fd-wrl6p"] Jul 24 13:15:53.045499 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.045472 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.049917 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.049876 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 13:15:53.050039 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.049948 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 13:15:53.082125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.082092 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z656h\"" Jul 24 13:15:53.082298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.082092 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 13:15:53.082298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.082178 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 13:15:53.082436 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.082371 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 13:15:53.084361 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.084335 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cd96db8fd-wrl6p"] Jul 24 13:15:53.089990 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.089960 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 13:15:53.138060 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138023 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-trusted-ca-bundle\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.138060 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138061 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-oauth-serving-cert\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.138268 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138097 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-service-ca\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.138268 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484e29c8-ae08-490f-b48e-2eeb17311591-console-serving-cert\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.138268 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138225 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b2m7\" (UniqueName: \"kubernetes.io/projected/484e29c8-ae08-490f-b48e-2eeb17311591-kube-api-access-8b2m7\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.138359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138285 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484e29c8-ae08-490f-b48e-2eeb17311591-console-oauth-config\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.138359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.138312 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-console-config\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.232473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.232358 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:53.239054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239020 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b2m7\" (UniqueName: \"kubernetes.io/projected/484e29c8-ae08-490f-b48e-2eeb17311591-kube-api-access-8b2m7\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239084 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484e29c8-ae08-490f-b48e-2eeb17311591-console-oauth-config\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239115 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-console-config\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239152 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-trusted-ca-bundle\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-oauth-serving-cert\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239459 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239230 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-service-ca\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239459 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239266 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484e29c8-ae08-490f-b48e-2eeb17311591-console-serving-cert\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.239927 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.239855 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-console-config\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.240045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.240009 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-service-ca\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.240045 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.240017 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-oauth-serving-cert\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.240202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.240181 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/484e29c8-ae08-490f-b48e-2eeb17311591-trusted-ca-bundle\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.242274 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.242252 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/484e29c8-ae08-490f-b48e-2eeb17311591-console-oauth-config\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.242520 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.242497 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/484e29c8-ae08-490f-b48e-2eeb17311591-console-serving-cert\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.256552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.256520 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b2m7\" (UniqueName: \"kubernetes.io/projected/484e29c8-ae08-490f-b48e-2eeb17311591-kube-api-access-8b2m7\") pod \"console-7cd96db8fd-wrl6p\" (UID: \"484e29c8-ae08-490f-b48e-2eeb17311591\") " pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.375786 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.375750 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" event={"ID":"5a6b3ff9-4e43-47e0-b518-d7f71fe75e30","Type":"ContainerStarted","Data":"0c3ff9e419431711000be5c6ebbdd9b061cf747d8809f73a1b15efa8413c58c2"} Jul 24 13:15:53.377424 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.377374 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" event={"ID":"16508327-9ed0-43db-bca7-75393fd4359c","Type":"ContainerStarted","Data":"3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716"} Jul 24 13:15:53.377575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.377536 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:15:53.411552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.411490 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" podStartSLOduration=1.117918199 podStartE2EDuration="5.411469419s" totalImagesPullingTime="4.29355122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:15:53.406791031 +0000 UTC m=+416.770715124" watchObservedRunningTime="2026-07-24 13:15:53.411469419 +0000 UTC m=+416.775393510" Jul 24 13:15:53.448778 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.448737 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:15:53.467996 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.467965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:53.610946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.610907 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cd96db8fd-wrl6p"] Jul 24 13:15:53.615515 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:15:53.615487 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484e29c8_ae08_490f_b48e_2eeb17311591.slice/crio-b61e7948122b155fdf255d8582c63f440292a8ab430bbb5729ddbe5c10a9f380 WatchSource:0}: Error finding container b61e7948122b155fdf255d8582c63f440292a8ab430bbb5729ddbe5c10a9f380: Status 404 returned error can't find the container with id b61e7948122b155fdf255d8582c63f440292a8ab430bbb5729ddbe5c10a9f380 Jul 24 13:15:53.702744 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.702706 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g"] Jul 24 13:15:53.707313 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.707290 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:15:53.712010 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.711963 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-p4w5j\"" Jul 24 13:15:53.712735 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.712328 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 24 13:15:53.736878 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.736794 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g"] Jul 24 13:15:53.745227 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.745154 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh4q4\" (UniqueName: \"kubernetes.io/projected/e2227c62-61a4-4e1b-b1ad-98e911742c42-kube-api-access-zh4q4\") pod \"dns-operator-controller-manager-864dd78c4c-7nd9g\" (UID: \"e2227c62-61a4-4e1b-b1ad-98e911742c42\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:15:53.846609 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.846510 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4q4\" (UniqueName: \"kubernetes.io/projected/e2227c62-61a4-4e1b-b1ad-98e911742c42-kube-api-access-zh4q4\") pod \"dns-operator-controller-manager-864dd78c4c-7nd9g\" (UID: \"e2227c62-61a4-4e1b-b1ad-98e911742c42\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:15:53.872881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.872815 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4q4\" (UniqueName: \"kubernetes.io/projected/e2227c62-61a4-4e1b-b1ad-98e911742c42-kube-api-access-zh4q4\") pod \"dns-operator-controller-manager-864dd78c4c-7nd9g\" (UID: \"e2227c62-61a4-4e1b-b1ad-98e911742c42\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:15:53.960506 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:53.960468 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:54.022086 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:54.022028 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:15:54.222557 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:54.222524 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g"] Jul 24 13:15:54.226676 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:15:54.226635 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2227c62_61a4_4e1b_b1ad_98e911742c42.slice/crio-50db5638377e553a359bc321dcc35bf27e7a433cb99cc6cde53a7155198bbacd WatchSource:0}: Error finding container 50db5638377e553a359bc321dcc35bf27e7a433cb99cc6cde53a7155198bbacd: Status 404 returned error can't find the container with id 50db5638377e553a359bc321dcc35bf27e7a433cb99cc6cde53a7155198bbacd Jul 24 13:15:54.384738 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:54.384648 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" event={"ID":"e2227c62-61a4-4e1b-b1ad-98e911742c42","Type":"ContainerStarted","Data":"50db5638377e553a359bc321dcc35bf27e7a433cb99cc6cde53a7155198bbacd"} Jul 24 13:15:54.386650 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:54.386614 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd96db8fd-wrl6p" event={"ID":"484e29c8-ae08-490f-b48e-2eeb17311591","Type":"ContainerStarted","Data":"1ad3d504d72b09311ad12230dff9a9260dae50fa578342f05f08e1a183f62361"} Jul 24 13:15:54.386786 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:54.386673 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd96db8fd-wrl6p" event={"ID":"484e29c8-ae08-490f-b48e-2eeb17311591","Type":"ContainerStarted","Data":"b61e7948122b155fdf255d8582c63f440292a8ab430bbb5729ddbe5c10a9f380"} Jul 24 13:15:54.412618 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:54.412559 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cd96db8fd-wrl6p" podStartSLOduration=1.412536528 podStartE2EDuration="1.412536528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:15:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:15:54.410576015 +0000 UTC m=+417.774500106" watchObservedRunningTime="2026-07-24 13:15:54.412536528 +0000 UTC m=+417.776460618" Jul 24 13:15:56.067441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:56.067377 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:56.399427 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:56.399363 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" event={"ID":"5a6b3ff9-4e43-47e0-b518-d7f71fe75e30","Type":"ContainerStarted","Data":"d0ffa4cc4fdd762ddc4de6e91bc3c62630484e911d771e5c6e118d0e747d9f3b"} Jul 24 13:15:56.399637 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:56.399453 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:15:56.420375 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:56.420312 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" podStartSLOduration=3.028470867 podStartE2EDuration="5.420287502s" totalImagesPullingTime="2.391816635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:15:56.41932622 +0000 UTC m=+419.783250312" watchObservedRunningTime="2026-07-24 13:15:56.420287502 +0000 UTC m=+419.784211593" Jul 24 13:15:56.732012 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:56.731969 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:56.896006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:56.895965 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:57.406251 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:57.406211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" event={"ID":"e2227c62-61a4-4e1b-b1ad-98e911742c42","Type":"ContainerStarted","Data":"372d034deb25eb965f5fb4c494934d9b482021c0a9e8a33c9c04d559d5620d29"} Jul 24 13:15:57.406745 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:57.406448 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:15:57.427347 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:57.427274 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" podStartSLOduration=2.000748177 podStartE2EDuration="4.427249637s" totalImagesPullingTime="2.42650146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:15:57.426140071 +0000 UTC m=+420.790064163" watchObservedRunningTime="2026-07-24 13:15:57.427249637 +0000 UTC m=+420.791173729" Jul 24 13:15:57.711410 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:57.711308 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:15:57.815399 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:15:57.815364 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:03.449638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:03.449604 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:16:03.450106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:03.449664 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:16:03.454640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:03.454617 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:16:04.389043 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:04.389007 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:04.446257 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:04.446224 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cd96db8fd-wrl6p" Jul 24 13:16:04.686613 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:04.686514 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:04.804552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:04.804511 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:06.451954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.451917 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j"] Jul 24 13:16:06.452377 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.452095 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" podUID="16508327-9ed0-43db-bca7-75393fd4359c" containerName="manager" containerID="cri-o://3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716" gracePeriod=2 Jul 24 13:16:06.456609 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.456573 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j"] Jul 24 13:16:06.467989 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:06.467949 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" is forbidden: User \"system:node:ip-10-0-139-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-210.ec2.internal' and this object" podUID="16508327-9ed0-43db-bca7-75393fd4359c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:06.486328 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.486295 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q"] Jul 24 13:16:06.486812 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.486795 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16508327-9ed0-43db-bca7-75393fd4359c" containerName="manager" Jul 24 13:16:06.486861 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.486815 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="16508327-9ed0-43db-bca7-75393fd4359c" containerName="manager" Jul 24 13:16:06.486907 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.486896 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="16508327-9ed0-43db-bca7-75393fd4359c" containerName="manager" Jul 24 13:16:06.489924 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.489902 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.526514 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.526475 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q"] Jul 24 13:16:06.531236 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.531203 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz"] Jul 24 13:16:06.535054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.535029 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:06.541946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.541918 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nv4vb\"" Jul 24 13:16:06.552365 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.552332 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz"] Jul 24 13:16:06.565052 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.565015 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b65f829-65cb-45ae-a6c4-264c2a9875e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r5s5q\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.565242 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.565105 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lslk9\" (UniqueName: \"kubernetes.io/projected/4b65f829-65cb-45ae-a6c4-264c2a9875e5-kube-api-access-lslk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r5s5q\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.593166 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:06.593121 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" is forbidden: User \"system:node:ip-10-0-139-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-210.ec2.internal' and this object" podUID="16508327-9ed0-43db-bca7-75393fd4359c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:06.623686 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:06.623641 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" is forbidden: User \"system:node:ip-10-0-139-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-210.ec2.internal' and this object" podUID="16508327-9ed0-43db-bca7-75393fd4359c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:06.665745 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.665692 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwtkp\" (UniqueName: \"kubernetes.io/projected/43ee53d1-c0c0-4237-aa70-605c9ec61d60-kube-api-access-wwtkp\") pod \"limitador-operator-controller-manager-859d7b78bd-snksz\" (UID: \"43ee53d1-c0c0-4237-aa70-605c9ec61d60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:06.665953 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.665765 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lslk9\" (UniqueName: \"kubernetes.io/projected/4b65f829-65cb-45ae-a6c4-264c2a9875e5-kube-api-access-lslk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r5s5q\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.665953 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.665843 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b65f829-65cb-45ae-a6c4-264c2a9875e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r5s5q\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.666250 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.666229 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b65f829-65cb-45ae-a6c4-264c2a9875e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r5s5q\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.708378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.708286 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lslk9\" (UniqueName: \"kubernetes.io/projected/4b65f829-65cb-45ae-a6c4-264c2a9875e5-kube-api-access-lslk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r5s5q\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.731650 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.731620 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:06.734246 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:06.734217 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" is forbidden: User \"system:node:ip-10-0-139-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-210.ec2.internal' and this object" podUID="16508327-9ed0-43db-bca7-75393fd4359c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:06.766805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.766769 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwtkp\" (UniqueName: \"kubernetes.io/projected/43ee53d1-c0c0-4237-aa70-605c9ec61d60-kube-api-access-wwtkp\") pod \"limitador-operator-controller-manager-859d7b78bd-snksz\" (UID: \"43ee53d1-c0c0-4237-aa70-605c9ec61d60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:06.784767 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.784729 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwtkp\" (UniqueName: \"kubernetes.io/projected/43ee53d1-c0c0-4237-aa70-605c9ec61d60-kube-api-access-wwtkp\") pod \"limitador-operator-controller-manager-859d7b78bd-snksz\" (UID: \"43ee53d1-c0c0-4237-aa70-605c9ec61d60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:06.835768 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.835721 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:06.845804 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.845775 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:06.867158 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.867125 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dzpz\" (UniqueName: \"kubernetes.io/projected/16508327-9ed0-43db-bca7-75393fd4359c-kube-api-access-8dzpz\") pod \"16508327-9ed0-43db-bca7-75393fd4359c\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " Jul 24 13:16:06.867322 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.867273 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16508327-9ed0-43db-bca7-75393fd4359c-extensions-socket-volume\") pod \"16508327-9ed0-43db-bca7-75393fd4359c\" (UID: \"16508327-9ed0-43db-bca7-75393fd4359c\") " Jul 24 13:16:06.867751 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.867705 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16508327-9ed0-43db-bca7-75393fd4359c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "16508327-9ed0-43db-bca7-75393fd4359c" (UID: "16508327-9ed0-43db-bca7-75393fd4359c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:16:06.869481 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.869457 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16508327-9ed0-43db-bca7-75393fd4359c-kube-api-access-8dzpz" (OuterVolumeSpecName: "kube-api-access-8dzpz") pod "16508327-9ed0-43db-bca7-75393fd4359c" (UID: "16508327-9ed0-43db-bca7-75393fd4359c"). InnerVolumeSpecName "kube-api-access-8dzpz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:16:06.969335 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.969125 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dzpz\" (UniqueName: \"kubernetes.io/projected/16508327-9ed0-43db-bca7-75393fd4359c-kube-api-access-8dzpz\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:16:06.969335 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:06.969165 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/16508327-9ed0-43db-bca7-75393fd4359c-extensions-socket-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:16:07.024785 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.024638 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q"] Jul 24 13:16:07.028117 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:16:07.028065 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b65f829_65cb_45ae_a6c4_264c2a9875e5.slice/crio-854603b1af8030dd08457051b020fea3ab7568663535ca2069dbb29b55685e74 WatchSource:0}: Error finding container 854603b1af8030dd08457051b020fea3ab7568663535ca2069dbb29b55685e74: Status 404 returned error can't find the container with id 854603b1af8030dd08457051b020fea3ab7568663535ca2069dbb29b55685e74 Jul 24 13:16:07.036624 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.036598 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz"] Jul 24 13:16:07.038573 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:16:07.038542 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43ee53d1_c0c0_4237_aa70_605c9ec61d60.slice/crio-30fd144ae832556d97c5f723568695394f7e56fa12cf416df84ba759e48a9b7a WatchSource:0}: Error finding container 30fd144ae832556d97c5f723568695394f7e56fa12cf416df84ba759e48a9b7a: Status 404 returned error can't find the container with id 30fd144ae832556d97c5f723568695394f7e56fa12cf416df84ba759e48a9b7a Jul 24 13:16:07.196286 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.196252 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16508327-9ed0-43db-bca7-75393fd4359c" path="/var/lib/kubelet/pods/16508327-9ed0-43db-bca7-75393fd4359c/volumes" Jul 24 13:16:07.198665 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:07.198633 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4t55j\" is forbidden: User \"system:node:ip-10-0-139-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-210.ec2.internal' and this object" podUID="16508327-9ed0-43db-bca7-75393fd4359c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:07.408652 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.408622 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xbgcb" Jul 24 13:16:07.459848 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.459808 2558 generic.go:356] "Generic (PLEG): container finished" podID="16508327-9ed0-43db-bca7-75393fd4359c" containerID="3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716" exitCode=0 Jul 24 13:16:07.460314 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.459964 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4t55j" Jul 24 13:16:07.460314 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.460074 2558 scope.go:117] "RemoveContainer" containerID="3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716" Jul 24 13:16:07.462950 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.462914 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" event={"ID":"4b65f829-65cb-45ae-a6c4-264c2a9875e5","Type":"ContainerStarted","Data":"2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4"} Jul 24 13:16:07.463080 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.462985 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" event={"ID":"4b65f829-65cb-45ae-a6c4-264c2a9875e5","Type":"ContainerStarted","Data":"854603b1af8030dd08457051b020fea3ab7568663535ca2069dbb29b55685e74"} Jul 24 13:16:07.463560 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.463322 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:07.464759 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.464732 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" event={"ID":"43ee53d1-c0c0-4237-aa70-605c9ec61d60","Type":"ContainerStarted","Data":"30fd144ae832556d97c5f723568695394f7e56fa12cf416df84ba759e48a9b7a"} Jul 24 13:16:07.497830 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.497774 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" podStartSLOduration=1.497752909 podStartE2EDuration="1.497752909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:16:07.497060409 +0000 UTC m=+430.860984500" watchObservedRunningTime="2026-07-24 13:16:07.497752909 +0000 UTC m=+430.861677001" Jul 24 13:16:07.523411 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.523361 2558 scope.go:117] "RemoveContainer" containerID="3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716" Jul 24 13:16:07.523777 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:07.523749 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716\": container with ID starting with 3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716 not found: ID does not exist" containerID="3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716" Jul 24 13:16:07.523879 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.523803 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716"} err="failed to get container status \"3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716\": rpc error: code = NotFound desc = could not find container \"3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716\": container with ID starting with 3b66f65e57d8046af1f3681c9c282fd1e3e5e0cdb67f1658a645dbe23a709716 not found: ID does not exist" Jul 24 13:16:07.859950 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.859910 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:07.997254 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:07.997213 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:08.414320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.414294 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7nd9g" Jul 24 13:16:08.472965 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.472915 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" event={"ID":"43ee53d1-c0c0-4237-aa70-605c9ec61d60","Type":"ContainerStarted","Data":"cb26d9261e06a0108aa9e094bd91413ffe990f117f7ef5e5b1557902841b6ff6"} Jul 24 13:16:08.473484 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.473075 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:08.507023 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.506962 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" podStartSLOduration=1.192355034 podStartE2EDuration="2.506946047s" totalImagesPullingTime="1.314591013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:16:08.505577118 +0000 UTC m=+431.869501210" watchObservedRunningTime="2026-07-24 13:16:08.506946047 +0000 UTC m=+431.870870138" Jul 24 13:16:08.625096 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.625014 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh"] Jul 24 13:16:08.628718 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.628698 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.662362 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.662324 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh"] Jul 24 13:16:08.687020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.686976 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x56wl\" (UniqueName: \"kubernetes.io/projected/50b7e2ee-6790-443f-873c-f303ff5a5555-kube-api-access-x56wl\") pod \"kuadrant-operator-controller-manager-86878c769f-c7ghh\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.687207 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.687134 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50b7e2ee-6790-443f-873c-f303ff5a5555-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7ghh\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.788308 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.788269 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50b7e2ee-6790-443f-873c-f303ff5a5555-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7ghh\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.788504 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.788337 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x56wl\" (UniqueName: \"kubernetes.io/projected/50b7e2ee-6790-443f-873c-f303ff5a5555-kube-api-access-x56wl\") pod \"kuadrant-operator-controller-manager-86878c769f-c7ghh\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.788718 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.788695 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50b7e2ee-6790-443f-873c-f303ff5a5555-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7ghh\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.820357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.820317 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x56wl\" (UniqueName: \"kubernetes.io/projected/50b7e2ee-6790-443f-873c-f303ff5a5555-kube-api-access-x56wl\") pod \"kuadrant-operator-controller-manager-86878c769f-c7ghh\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:08.939263 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:08.939157 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:09.109990 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:09.109962 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh"] Jul 24 13:16:09.112949 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:16:09.112911 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b7e2ee_6790_443f_873c_f303ff5a5555.slice/crio-d0e89635cafbc784ee08285419ed507d25ed500bce5627cc537988e8e659a3bc WatchSource:0}: Error finding container d0e89635cafbc784ee08285419ed507d25ed500bce5627cc537988e8e659a3bc: Status 404 returned error can't find the container with id d0e89635cafbc784ee08285419ed507d25ed500bce5627cc537988e8e659a3bc Jul 24 13:16:09.480574 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:09.480483 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" event={"ID":"50b7e2ee-6790-443f-873c-f303ff5a5555","Type":"ContainerStarted","Data":"20cfd291d5a44ac2a0f9aeccfa36ae1667e19ab0650ebe24fbe790b285fbe8fa"} Jul 24 13:16:09.480574 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:09.480543 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" event={"ID":"50b7e2ee-6790-443f-873c-f303ff5a5555","Type":"ContainerStarted","Data":"d0e89635cafbc784ee08285419ed507d25ed500bce5627cc537988e8e659a3bc"} Jul 24 13:16:09.480985 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:09.480770 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:09.518631 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:09.518551 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" podStartSLOduration=1.5185249029999999 podStartE2EDuration="1.518524903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:16:09.516617782 +0000 UTC m=+432.880541940" watchObservedRunningTime="2026-07-24 13:16:09.518524903 +0000 UTC m=+432.882449066" Jul 24 13:16:10.308737 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:10.308698 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:10.418085 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:10.418054 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:14.940263 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:14.940224 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:15.043677 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:15.043634 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:18.477243 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:18.477213 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:18.741746 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:18.741641 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:18.843189 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:18.843150 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:19.483599 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:19.483513 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-snksz" Jul 24 13:16:20.488638 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.488605 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:16:20.546741 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.546696 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q"] Jul 24 13:16:20.546976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.546934 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" podUID="4b65f829-65cb-45ae-a6c4-264c2a9875e5" containerName="manager" containerID="cri-o://2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4" gracePeriod=10 Jul 24 13:16:20.813585 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.813556 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:20.907238 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.907198 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lslk9\" (UniqueName: \"kubernetes.io/projected/4b65f829-65cb-45ae-a6c4-264c2a9875e5-kube-api-access-lslk9\") pod \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " Jul 24 13:16:20.907451 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.907332 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b65f829-65cb-45ae-a6c4-264c2a9875e5-extensions-socket-volume\") pod \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\" (UID: \"4b65f829-65cb-45ae-a6c4-264c2a9875e5\") " Jul 24 13:16:20.907761 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.907729 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b65f829-65cb-45ae-a6c4-264c2a9875e5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b65f829-65cb-45ae-a6c4-264c2a9875e5" (UID: "4b65f829-65cb-45ae-a6c4-264c2a9875e5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:16:20.909258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:20.909231 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b65f829-65cb-45ae-a6c4-264c2a9875e5-kube-api-access-lslk9" (OuterVolumeSpecName: "kube-api-access-lslk9") pod "4b65f829-65cb-45ae-a6c4-264c2a9875e5" (UID: "4b65f829-65cb-45ae-a6c4-264c2a9875e5"). InnerVolumeSpecName "kube-api-access-lslk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:16:21.008959 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.008910 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b65f829-65cb-45ae-a6c4-264c2a9875e5-extensions-socket-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:16:21.008959 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.008950 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lslk9\" (UniqueName: \"kubernetes.io/projected/4b65f829-65cb-45ae-a6c4-264c2a9875e5-kube-api-access-lslk9\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:16:21.546252 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.546148 2558 generic.go:356] "Generic (PLEG): container finished" podID="4b65f829-65cb-45ae-a6c4-264c2a9875e5" containerID="2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4" exitCode=0 Jul 24 13:16:21.546252 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.546183 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" event={"ID":"4b65f829-65cb-45ae-a6c4-264c2a9875e5","Type":"ContainerDied","Data":"2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4"} Jul 24 13:16:21.546252 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.546199 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" Jul 24 13:16:21.546834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.546254 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q" event={"ID":"4b65f829-65cb-45ae-a6c4-264c2a9875e5","Type":"ContainerDied","Data":"854603b1af8030dd08457051b020fea3ab7568663535ca2069dbb29b55685e74"} Jul 24 13:16:21.546834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.546289 2558 scope.go:117] "RemoveContainer" containerID="2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4" Jul 24 13:16:21.555659 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.555523 2558 scope.go:117] "RemoveContainer" containerID="2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4" Jul 24 13:16:21.555826 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:16:21.555804 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4\": container with ID starting with 2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4 not found: ID does not exist" containerID="2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4" Jul 24 13:16:21.555871 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.555836 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4"} err="failed to get container status \"2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4\": rpc error: code = NotFound desc = could not find container \"2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4\": container with ID starting with 2a64dab76712e4cf326b5b88ddb369bf0dde7f9dd35349c0eda596f2f678dcf4 not found: ID does not exist" Jul 24 13:16:21.565103 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.565071 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q"] Jul 24 13:16:21.578070 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:21.578034 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r5s5q"] Jul 24 13:16:23.196776 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:23.196736 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b65f829-65cb-45ae-a6c4-264c2a9875e5" path="/var/lib/kubelet/pods/4b65f829-65cb-45ae-a6c4-264c2a9875e5/volumes" Jul 24 13:16:27.494311 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:27.494261 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:27.598034 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:27.597998 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:29.947676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:29.947640 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:30.048677 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:30.048634 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:34.689884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:34.689843 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:34.793108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:34.793072 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:37.053979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:37.053943 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:37.440844 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:37.440794 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:38.704503 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:38.704470 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:38.868575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:38.868537 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:38.975425 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:38.975312 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:39.143762 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.143712 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:39.164512 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.164476 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:16:39.164905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.164893 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b65f829-65cb-45ae-a6c4-264c2a9875e5" containerName="manager" Jul 24 13:16:39.164949 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.164907 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b65f829-65cb-45ae-a6c4-264c2a9875e5" containerName="manager" Jul 24 13:16:39.164992 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.164984 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b65f829-65cb-45ae-a6c4-264c2a9875e5" containerName="manager" Jul 24 13:16:39.170515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.170488 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.173161 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.173139 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 24 13:16:39.173349 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.173325 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v9v7d\"" Jul 24 13:16:39.185822 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.185791 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:16:39.199063 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.199030 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:16:39.231615 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.231520 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:39.250748 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.250708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s74l\" (UniqueName: \"kubernetes.io/projected/7d6a8e07-7178-4469-b1c2-4706602e12f7-kube-api-access-8s74l\") pod \"limitador-limitador-54d6d988d5-zz7mw\" (UID: \"7d6a8e07-7178-4469-b1c2-4706602e12f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.250918 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.250761 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d6a8e07-7178-4469-b1c2-4706602e12f7-config-file\") pod \"limitador-limitador-54d6d988d5-zz7mw\" (UID: \"7d6a8e07-7178-4469-b1c2-4706602e12f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.352083 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.352044 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s74l\" (UniqueName: \"kubernetes.io/projected/7d6a8e07-7178-4469-b1c2-4706602e12f7-kube-api-access-8s74l\") pod \"limitador-limitador-54d6d988d5-zz7mw\" (UID: \"7d6a8e07-7178-4469-b1c2-4706602e12f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.352274 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.352105 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d6a8e07-7178-4469-b1c2-4706602e12f7-config-file\") pod \"limitador-limitador-54d6d988d5-zz7mw\" (UID: \"7d6a8e07-7178-4469-b1c2-4706602e12f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.352859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.352829 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d6a8e07-7178-4469-b1c2-4706602e12f7-config-file\") pod \"limitador-limitador-54d6d988d5-zz7mw\" (UID: \"7d6a8e07-7178-4469-b1c2-4706602e12f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.360757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.360730 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s74l\" (UniqueName: \"kubernetes.io/projected/7d6a8e07-7178-4469-b1c2-4706602e12f7-kube-api-access-8s74l\") pod \"limitador-limitador-54d6d988d5-zz7mw\" (UID: \"7d6a8e07-7178-4469-b1c2-4706602e12f7\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.481949 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.481824 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:39.632756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.632639 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:16:39.636126 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:16:39.636090 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6a8e07_7178_4469_b1c2_4706602e12f7.slice/crio-26f3fc4f13e4ea7705ca08169f62c89cbf699b4099f9b5f20633b287a9aa235c WatchSource:0}: Error finding container 26f3fc4f13e4ea7705ca08169f62c89cbf699b4099f9b5f20633b287a9aa235c: Status 404 returned error can't find the container with id 26f3fc4f13e4ea7705ca08169f62c89cbf699b4099f9b5f20633b287a9aa235c Jul 24 13:16:39.646267 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.646239 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" event={"ID":"7d6a8e07-7178-4469-b1c2-4706602e12f7","Type":"ContainerStarted","Data":"26f3fc4f13e4ea7705ca08169f62c89cbf699b4099f9b5f20633b287a9aa235c"} Jul 24 13:16:39.835425 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.835319 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:39.946087 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:39.946049 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:42.665115 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:42.665076 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" event={"ID":"7d6a8e07-7178-4469-b1c2-4706602e12f7","Type":"ContainerStarted","Data":"36dc831368e1a0e25ae3a7e7ecda406429f6094b3055f1743526220305f8954c"} Jul 24 13:16:42.665572 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:42.665149 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:42.685860 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:42.685795 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" podStartSLOduration=1.121079003 podStartE2EDuration="3.685771462s" totalImagesPullingTime="2.564692459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:16:42.68415153 +0000 UTC m=+466.048075628" watchObservedRunningTime="2026-07-24 13:16:42.685771462 +0000 UTC m=+466.049695555" Jul 24 13:16:48.902819 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:48.902785 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:48.998999 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:48.998961 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:49.100718 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:49.100673 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:49.217146 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:49.217028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:50.695446 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:50.695406 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:50.799700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:50.799653 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:53.671218 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:53.671183 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-zz7mw" Jul 24 13:16:55.180944 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:55.180908 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:55.352682 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:55.352648 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:55.681773 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:55.681732 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:16:55.781415 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:16:55.781349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:07.662030 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:07.661991 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:07.865975 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:07.865915 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:15.237747 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:15.237699 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:15.346563 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:15.346529 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:15.426612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:15.426556 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:18.493961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:18.493920 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:18.594140 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:18.594101 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:35.404457 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:35.404381 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:51.079037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:51.078924 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:51.176836 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:51.176798 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:51.712993 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:51.712954 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:52.000057 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:51.999954 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:52.220820 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:52.220780 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:52.323004 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:52.322908 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:17:52.397901 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:17:52.397847 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:18:02.217673 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:02.217617 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:02.332983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:02.332946 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:08.071465 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:08.071417 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:12.652432 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:12.652345 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:13.119105 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:13.119072 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:18:17.701524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:17.701489 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:17.802280 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:17.802241 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:31.905303 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:31.905246 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:32.823651 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:32.823603 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:18:39.747852 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:39.747800 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:40.945797 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:40.945762 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:18:48.396730 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:48.396690 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:18:57.062809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:57.062774 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:18:57.064364 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:18:57.064339 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:19:03.314503 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.314454 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g"] Jul 24 13:19:03.318031 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.318016 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.326770 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.326736 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 13:19:03.327037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.327017 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 24 13:19:03.327298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.327273 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 13:19:03.327495 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.327473 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4v65n\"" Jul 24 13:19:03.329870 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.329845 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g"] Jul 24 13:19:03.416997 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.416921 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:19:03.449094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.449060 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.449300 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.449115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzl5c\" (UniqueName: \"kubernetes.io/projected/48ab9ba5-ab8b-475a-9771-9a2928b976de-kube-api-access-kzl5c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.449300 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.449171 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.449300 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.449205 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.449518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.449302 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48ab9ba5-ab8b-475a-9771-9a2928b976de-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.449518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.449343 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.550524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550481 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.550740 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550537 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.550740 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550583 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48ab9ba5-ab8b-475a-9771-9a2928b976de-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.550740 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550603 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.550740 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550638 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.550740 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550689 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzl5c\" (UniqueName: \"kubernetes.io/projected/48ab9ba5-ab8b-475a-9771-9a2928b976de-kube-api-access-kzl5c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.551008 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550925 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.551008 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.550980 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.551134 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.551106 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.552914 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.552892 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48ab9ba5-ab8b-475a-9771-9a2928b976de-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.553283 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.553262 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48ab9ba5-ab8b-475a-9771-9a2928b976de-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.558917 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.558883 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzl5c\" (UniqueName: \"kubernetes.io/projected/48ab9ba5-ab8b-475a-9771-9a2928b976de-kube-api-access-kzl5c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g\" (UID: \"48ab9ba5-ab8b-475a-9771-9a2928b976de\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.635163 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.635121 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:03.773218 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.773187 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g"] Jul 24 13:19:03.775824 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:19:03.775793 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ab9ba5_ab8b_475a_9771_9a2928b976de.slice/crio-0dc380836d8b1e967c2ac7d7757e6a258ab2fcdd030264a1aeee3622162fe898 WatchSource:0}: Error finding container 0dc380836d8b1e967c2ac7d7757e6a258ab2fcdd030264a1aeee3622162fe898: Status 404 returned error can't find the container with id 0dc380836d8b1e967c2ac7d7757e6a258ab2fcdd030264a1aeee3622162fe898 Jul 24 13:19:03.777178 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:03.777159 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:19:04.396118 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:04.396083 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" event={"ID":"48ab9ba5-ab8b-475a-9771-9a2928b976de","Type":"ContainerStarted","Data":"0dc380836d8b1e967c2ac7d7757e6a258ab2fcdd030264a1aeee3622162fe898"} Jul 24 13:19:05.212546 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:05.212512 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:19:05.402771 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:05.402734 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" event={"ID":"48ab9ba5-ab8b-475a-9771-9a2928b976de","Type":"ContainerStarted","Data":"77c5ff68bb041a26002d6ac148e6d9c12c879c69e7522d6b586ce977c402edb7"} Jul 24 13:19:05.420084 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:05.420015 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" podStartSLOduration=1.431413223 podStartE2EDuration="2.41999953s" totalImagesPullingTime="988.586307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:19:05.419134474 +0000 UTC m=+608.783058575" watchObservedRunningTime="2026-07-24 13:19:05.41999953 +0000 UTC m=+608.783923621" Jul 24 13:19:10.735136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:10.735094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:19:13.608758 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:13.608720 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:19:13.635939 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:13.635903 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:13.636106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:13.635965 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:13.648848 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:13.648810 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:14.458610 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:14.458576 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g" Jul 24 13:19:15.506721 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:15.506674 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:19:25.519722 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:25.519682 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:19:43.456321 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:43.456276 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:19:57.021631 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:19:57.021591 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:20:25.145295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:20:25.145260 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:20:25.253454 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:20:25.253417 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:20:26.420890 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:20:26.420854 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:20:26.519394 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:20:26.519359 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:20:58.122392 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:20:58.122296 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:21:07.816324 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:21:07.816284 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:21:18.426323 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:21:18.426283 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:22:15.253676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:15.253638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:22:15.309067 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:15.309039 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:22:22.387166 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:22.387090 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:22:22.486561 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:22.486522 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:22:34.102550 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:34.102514 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:22:35.342203 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:35.342170 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:22:59.519126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:22:59.519095 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:23:10.214518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:10.214471 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:23:18.275573 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:18.275542 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:18.380603 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:18.380570 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:18.536688 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:18.536607 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:25.024649 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:25.024612 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:25.125949 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:25.125917 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:25.224723 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:25.224681 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:25.422648 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:25.422614 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:31.147558 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:31.147518 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:31.246940 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:31.246908 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:23:48.419462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:48.419419 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:23:57.093564 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:57.093490 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:23:57.096302 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:57.096281 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:23:57.716723 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:23:57.716688 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:24:07.514986 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:24:07.514942 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:24:15.307373 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:24:15.307338 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:24:25.621936 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:24:25.621904 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:24:35.819834 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:24:35.819796 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:24:47.117184 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:24:47.117148 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:24:56.904654 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:24:56.904614 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:25:19.410292 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:25:19.410219 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:25:32.305092 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:25:32.305058 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:25:54.711804 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:25:54.711771 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:26:04.204799 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:26:04.204761 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:26:21.519089 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:26:21.519055 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:26:30.614024 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:26:30.613989 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:26:41.977898 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:26:41.977866 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:26:50.822103 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:26:50.822027 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:26:59.316704 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:26:59.316673 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:27:26.555445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:26.555410 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh"] Jul 24 13:27:26.555946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:26.555649 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" podUID="50b7e2ee-6790-443f-873c-f303ff5a5555" containerName="manager" containerID="cri-o://20cfd291d5a44ac2a0f9aeccfa36ae1667e19ab0650ebe24fbe790b285fbe8fa" gracePeriod=10 Jul 24 13:27:28.858320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.858288 2558 generic.go:356] "Generic (PLEG): container finished" podID="50b7e2ee-6790-443f-873c-f303ff5a5555" containerID="20cfd291d5a44ac2a0f9aeccfa36ae1667e19ab0650ebe24fbe790b285fbe8fa" exitCode=0 Jul 24 13:27:28.858669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.858354 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" event={"ID":"50b7e2ee-6790-443f-873c-f303ff5a5555","Type":"ContainerDied","Data":"20cfd291d5a44ac2a0f9aeccfa36ae1667e19ab0650ebe24fbe790b285fbe8fa"} Jul 24 13:27:28.906196 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.906172 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:27:28.948720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.948693 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x56wl\" (UniqueName: \"kubernetes.io/projected/50b7e2ee-6790-443f-873c-f303ff5a5555-kube-api-access-x56wl\") pod \"50b7e2ee-6790-443f-873c-f303ff5a5555\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " Jul 24 13:27:28.948855 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.948747 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50b7e2ee-6790-443f-873c-f303ff5a5555-extensions-socket-volume\") pod \"50b7e2ee-6790-443f-873c-f303ff5a5555\" (UID: \"50b7e2ee-6790-443f-873c-f303ff5a5555\") " Jul 24 13:27:28.949097 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.949076 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50b7e2ee-6790-443f-873c-f303ff5a5555-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "50b7e2ee-6790-443f-873c-f303ff5a5555" (UID: "50b7e2ee-6790-443f-873c-f303ff5a5555"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:27:28.950664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:28.950641 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50b7e2ee-6790-443f-873c-f303ff5a5555-kube-api-access-x56wl" (OuterVolumeSpecName: "kube-api-access-x56wl") pod "50b7e2ee-6790-443f-873c-f303ff5a5555" (UID: "50b7e2ee-6790-443f-873c-f303ff5a5555"). InnerVolumeSpecName "kube-api-access-x56wl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:27:29.049245 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.049198 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x56wl\" (UniqueName: \"kubernetes.io/projected/50b7e2ee-6790-443f-873c-f303ff5a5555-kube-api-access-x56wl\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:27:29.049245 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.049220 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50b7e2ee-6790-443f-873c-f303ff5a5555-extensions-socket-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:27:29.864193 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.864165 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" Jul 24 13:27:29.864541 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.864189 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh" event={"ID":"50b7e2ee-6790-443f-873c-f303ff5a5555","Type":"ContainerDied","Data":"d0e89635cafbc784ee08285419ed507d25ed500bce5627cc537988e8e659a3bc"} Jul 24 13:27:29.864541 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.864264 2558 scope.go:117] "RemoveContainer" containerID="20cfd291d5a44ac2a0f9aeccfa36ae1667e19ab0650ebe24fbe790b285fbe8fa" Jul 24 13:27:29.882327 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.882304 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh"] Jul 24 13:27:29.885668 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:29.885647 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7ghh"] Jul 24 13:27:31.197200 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:27:31.197166 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50b7e2ee-6790-443f-873c-f303ff5a5555" path="/var/lib/kubelet/pods/50b7e2ee-6790-443f-873c-f303ff5a5555/volumes" Jul 24 13:28:02.619212 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:02.619178 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:28:27.779781 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.779748 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts"] Jul 24 13:28:27.780141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.780127 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50b7e2ee-6790-443f-873c-f303ff5a5555" containerName="manager" Jul 24 13:28:27.780141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.780139 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="50b7e2ee-6790-443f-873c-f303ff5a5555" containerName="manager" Jul 24 13:28:27.780215 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.780205 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="50b7e2ee-6790-443f-873c-f303ff5a5555" containerName="manager" Jul 24 13:28:27.783108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.783093 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:27.785655 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.785634 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mrhqc\"" Jul 24 13:28:27.795043 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.795023 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts"] Jul 24 13:28:27.871681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.871655 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d25266b-2a4c-4834-99bb-488ba55828c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ckvts\" (UID: \"4d25266b-2a4c-4834-99bb-488ba55828c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:27.871786 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.871690 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmklr\" (UniqueName: \"kubernetes.io/projected/4d25266b-2a4c-4834-99bb-488ba55828c3-kube-api-access-cmklr\") pod \"kuadrant-operator-controller-manager-86878c769f-ckvts\" (UID: \"4d25266b-2a4c-4834-99bb-488ba55828c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:27.972253 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.972232 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d25266b-2a4c-4834-99bb-488ba55828c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ckvts\" (UID: \"4d25266b-2a4c-4834-99bb-488ba55828c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:27.972359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.972261 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmklr\" (UniqueName: \"kubernetes.io/projected/4d25266b-2a4c-4834-99bb-488ba55828c3-kube-api-access-cmklr\") pod \"kuadrant-operator-controller-manager-86878c769f-ckvts\" (UID: \"4d25266b-2a4c-4834-99bb-488ba55828c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:27.972625 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.972605 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d25266b-2a4c-4834-99bb-488ba55828c3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ckvts\" (UID: \"4d25266b-2a4c-4834-99bb-488ba55828c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:27.984339 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:27.984320 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmklr\" (UniqueName: \"kubernetes.io/projected/4d25266b-2a4c-4834-99bb-488ba55828c3-kube-api-access-cmklr\") pod \"kuadrant-operator-controller-manager-86878c769f-ckvts\" (UID: \"4d25266b-2a4c-4834-99bb-488ba55828c3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:28.093844 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:28.093792 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:28.231909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:28.231886 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts"] Jul 24 13:28:28.234445 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:28:28.234415 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d25266b_2a4c_4834_99bb_488ba55828c3.slice/crio-8b4933564183fa0bc6d6845019fa209fef33cfaab3191f7e2fd2a0af65435695 WatchSource:0}: Error finding container 8b4933564183fa0bc6d6845019fa209fef33cfaab3191f7e2fd2a0af65435695: Status 404 returned error can't find the container with id 8b4933564183fa0bc6d6845019fa209fef33cfaab3191f7e2fd2a0af65435695 Jul 24 13:28:29.145413 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:29.145355 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" event={"ID":"4d25266b-2a4c-4834-99bb-488ba55828c3","Type":"ContainerStarted","Data":"68963ec44b37747d43fea562eb587ff4950ab812bfbcbaed0b72acfccfe3b969"} Jul 24 13:28:29.145829 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:29.145424 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" event={"ID":"4d25266b-2a4c-4834-99bb-488ba55828c3","Type":"ContainerStarted","Data":"8b4933564183fa0bc6d6845019fa209fef33cfaab3191f7e2fd2a0af65435695"} Jul 24 13:28:29.145829 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:29.145505 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:29.170512 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:29.170465 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" podStartSLOduration=2.170452134 podStartE2EDuration="2.170452134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:28:29.168040711 +0000 UTC m=+1172.531964831" watchObservedRunningTime="2026-07-24 13:28:29.170452134 +0000 UTC m=+1172.534376203" Jul 24 13:28:40.152709 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:40.152681 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ckvts" Jul 24 13:28:49.928522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:49.928484 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:28:57.123432 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:57.123379 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:28:57.127147 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:28:57.127124 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:29:00.944735 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:00.944702 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:29:12.222025 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:12.221989 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:29:24.416056 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:24.416023 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:29:34.410829 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:34.410791 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:29:43.815930 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:43.815896 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:29:48.195179 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:48.195147 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:29:53.827010 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:29:53.826931 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:30:00.142983 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.142942 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz"] Jul 24 13:30:00.146640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.146618 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" Jul 24 13:30:00.148966 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.148944 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j482l\"" Jul 24 13:30:00.152325 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.152302 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz"] Jul 24 13:30:00.158334 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.158304 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" Jul 24 13:30:00.289102 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.289081 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz"] Jul 24 13:30:00.291236 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:30:00.291200 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1838dcd8_9fba_47be_a692_86002cbb6811.slice/crio-d7ad9db61de92bead84ee7cfe027ec00f301f299a309b06f09f8db2eedb2d59b WatchSource:0}: Error finding container d7ad9db61de92bead84ee7cfe027ec00f301f299a309b06f09f8db2eedb2d59b: Status 404 returned error can't find the container with id d7ad9db61de92bead84ee7cfe027ec00f301f299a309b06f09f8db2eedb2d59b Jul 24 13:30:00.293091 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.293073 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:30:00.584361 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:00.584294 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" event={"ID":"1838dcd8-9fba-47be-a692-86002cbb6811","Type":"ContainerStarted","Data":"d7ad9db61de92bead84ee7cfe027ec00f301f299a309b06f09f8db2eedb2d59b"} Jul 24 13:30:02.596201 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:02.596176 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" event={"ID":"1838dcd8-9fba-47be-a692-86002cbb6811","Type":"ContainerStarted","Data":"7c8d2cd7bc6aa74fd01e8555b3531e06e09629b7d633287b557a023facfc912d"} Jul 24 13:30:02.610969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:02.610930 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" podStartSLOduration=0.506379561 podStartE2EDuration="2.610915942s" totalImagesPullingTime="2.104536381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:30:02.610018765 +0000 UTC m=+1265.973942854" watchObservedRunningTime="2026-07-24 13:30:02.610915942 +0000 UTC m=+1265.974840033" Jul 24 13:30:03.601549 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:03.601516 2558 generic.go:356] "Generic (PLEG): container finished" podID="1838dcd8-9fba-47be-a692-86002cbb6811" containerID="7c8d2cd7bc6aa74fd01e8555b3531e06e09629b7d633287b557a023facfc912d" exitCode=0 Jul 24 13:30:03.601847 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:03.601565 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" event={"ID":"1838dcd8-9fba-47be-a692-86002cbb6811","Type":"ContainerDied","Data":"7c8d2cd7bc6aa74fd01e8555b3531e06e09629b7d633287b557a023facfc912d"} Jul 24 13:30:04.740504 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:04.740479 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" Jul 24 13:30:04.915072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:04.915000 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:30:05.612429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:05.612398 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" Jul 24 13:30:05.612429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:05.612422 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748330-hvbmz" event={"ID":"1838dcd8-9fba-47be-a692-86002cbb6811","Type":"ContainerDied","Data":"d7ad9db61de92bead84ee7cfe027ec00f301f299a309b06f09f8db2eedb2d59b"} Jul 24 13:30:05.612663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:05.612488 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7ad9db61de92bead84ee7cfe027ec00f301f299a309b06f09f8db2eedb2d59b" Jul 24 13:30:15.906605 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:15.906566 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:30:24.617321 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:24.617286 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:30:34.710442 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:30:34.710410 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:31:00.209261 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:00.209225 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:31:08.008127 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:08.008086 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:31:12.587715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:12.587674 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:31:23.664881 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:23.664848 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:31:26.617918 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:26.617882 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:31:28.247011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:28.246973 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:31:40.639325 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:40.639288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:31:40.740079 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:40.740047 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:31:41.013506 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:31:41.013426 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:32:01.113510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:01.113472 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:32:02.001178 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:02.001148 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:32:04.916246 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:04.916210 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:32:19.713465 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:19.713433 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:32:26.011800 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:26.011767 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:32:42.410845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:42.410803 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:32:53.109530 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:32:53.109451 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:02.809228 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:02.809193 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:11.613573 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:11.613537 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:21.511980 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:21.511940 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:31.112819 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:31.112785 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:44.417813 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:44.417771 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:53.613337 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:53.613294 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:33:57.155706 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:57.155669 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:33:57.162014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:33:57.161985 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:34:04.016343 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:34:04.016308 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:34:13.220333 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:34:13.220291 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:34:33.515737 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:34:33.515694 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:34:46.175592 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:34:46.175547 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:34:59.315935 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:34:59.315897 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:35:08.821462 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:08.821424 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:35:12.436159 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:12.436122 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:23.121762 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:23.121721 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:35:32.821362 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:32.821319 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:35:34.312620 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:34.312573 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:37.976215 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:37.976144 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:38.064686 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:38.064651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:38.433670 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:38.433641 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:42.114370 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:42.114328 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:35:44.446156 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:44.446124 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:44.545758 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:44.545723 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:44.698020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:44.697921 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:45.053901 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:45.053813 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:45.260606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:45.260570 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:47.999718 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:47.999676 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:48.093019 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.092974 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:48.197264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.197229 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:48.297442 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.297343 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:48.735509 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.735476 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:48.833369 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.833334 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:35:48.890047 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.889999 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d"] Jul 24 13:35:48.890438 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.890417 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1838dcd8-9fba-47be-a692-86002cbb6811" containerName="cleanup" Jul 24 13:35:48.890438 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.890438 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1838dcd8-9fba-47be-a692-86002cbb6811" containerName="cleanup" Jul 24 13:35:48.890602 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.890539 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1838dcd8-9fba-47be-a692-86002cbb6811" containerName="cleanup" Jul 24 13:35:48.893859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.893838 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:48.896206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.896184 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-77101517-dockercfg-bqhw8\"" Jul 24 13:35:48.896302 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.896189 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-77101517\"" Jul 24 13:35:48.905037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:48.905010 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d"] Jul 24 13:35:49.029603 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.029503 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kbng\" (UniqueName: \"kubernetes.io/projected/09af388a-8a77-4a22-9694-a1f05fe0af6d-kube-api-access-5kbng\") pod \"payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.029986 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.029638 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09af388a-8a77-4a22-9694-a1f05fe0af6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.130448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.130381 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09af388a-8a77-4a22-9694-a1f05fe0af6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.130646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.130474 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kbng\" (UniqueName: \"kubernetes.io/projected/09af388a-8a77-4a22-9694-a1f05fe0af6d-kube-api-access-5kbng\") pod \"payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.131088 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.131064 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09af388a-8a77-4a22-9694-a1f05fe0af6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.138996 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.138973 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kbng\" (UniqueName: \"kubernetes.io/projected/09af388a-8a77-4a22-9694-a1f05fe0af6d-kube-api-access-5kbng\") pod \"payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.206007 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.205978 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:49.557196 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.557169 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d"] Jul 24 13:35:49.560395 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:35:49.560356 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09af388a_8a77_4a22_9694_a1f05fe0af6d.slice/crio-14f39c487ad662b852995a62168ac94fe33b430afae045b1739b556f49459477 WatchSource:0}: Error finding container 14f39c487ad662b852995a62168ac94fe33b430afae045b1739b556f49459477: Status 404 returned error can't find the container with id 14f39c487ad662b852995a62168ac94fe33b430afae045b1739b556f49459477 Jul 24 13:35:49.562489 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:49.562466 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:35:50.335598 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:50.335556 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" event={"ID":"09af388a-8a77-4a22-9694-a1f05fe0af6d","Type":"ContainerStarted","Data":"14f39c487ad662b852995a62168ac94fe33b430afae045b1739b556f49459477"} Jul 24 13:35:52.350584 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:52.350546 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" event={"ID":"09af388a-8a77-4a22-9694-a1f05fe0af6d","Type":"ContainerStarted","Data":"fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da"} Jul 24 13:35:52.351005 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:52.350701 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:35:52.369106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:52.369051 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" podStartSLOduration=2.156026668 podStartE2EDuration="4.369035744s" totalImagesPullingTime="2.213009076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:52.366874466 +0000 UTC m=+1615.730798559" watchObservedRunningTime="2026-07-24 13:35:52.369035744 +0000 UTC m=+1615.732959834" Jul 24 13:35:55.711500 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:35:55.711461 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:36:03.359224 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:03.359191 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:36:24.038571 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.038526 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:24.077522 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.077481 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d"] Jul 24 13:36:24.077822 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.077760 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" podUID="09af388a-8a77-4a22-9694-a1f05fe0af6d" containerName="payload-pre-processing" containerID="cri-o://fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da" gracePeriod=30 Jul 24 13:36:24.138169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.138132 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:24.235969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.235933 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:24.329809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.329780 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:36:24.445626 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.445596 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kbng\" (UniqueName: \"kubernetes.io/projected/09af388a-8a77-4a22-9694-a1f05fe0af6d-kube-api-access-5kbng\") pod \"09af388a-8a77-4a22-9694-a1f05fe0af6d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " Jul 24 13:36:24.445823 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.445696 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09af388a-8a77-4a22-9694-a1f05fe0af6d-plugins-config-volume\") pod \"09af388a-8a77-4a22-9694-a1f05fe0af6d\" (UID: \"09af388a-8a77-4a22-9694-a1f05fe0af6d\") " Jul 24 13:36:24.446074 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.446051 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09af388a-8a77-4a22-9694-a1f05fe0af6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09af388a-8a77-4a22-9694-a1f05fe0af6d" (UID: "09af388a-8a77-4a22-9694-a1f05fe0af6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:36:24.447849 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.447818 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09af388a-8a77-4a22-9694-a1f05fe0af6d-kube-api-access-5kbng" (OuterVolumeSpecName: "kube-api-access-5kbng") pod "09af388a-8a77-4a22-9694-a1f05fe0af6d" (UID: "09af388a-8a77-4a22-9694-a1f05fe0af6d"). InnerVolumeSpecName "kube-api-access-5kbng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:36:24.514583 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.514550 2558 generic.go:356] "Generic (PLEG): container finished" podID="09af388a-8a77-4a22-9694-a1f05fe0af6d" containerID="fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da" exitCode=0 Jul 24 13:36:24.514769 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.514596 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" event={"ID":"09af388a-8a77-4a22-9694-a1f05fe0af6d","Type":"ContainerDied","Data":"fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da"} Jul 24 13:36:24.514769 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.514601 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" Jul 24 13:36:24.514769 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.514638 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d" event={"ID":"09af388a-8a77-4a22-9694-a1f05fe0af6d","Type":"ContainerDied","Data":"14f39c487ad662b852995a62168ac94fe33b430afae045b1739b556f49459477"} Jul 24 13:36:24.514769 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.514662 2558 scope.go:117] "RemoveContainer" containerID="fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da" Jul 24 13:36:24.527397 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.527366 2558 scope.go:117] "RemoveContainer" containerID="fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da" Jul 24 13:36:24.527672 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:36:24.527654 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da\": container with ID starting with fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da not found: ID does not exist" containerID="fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da" Jul 24 13:36:24.527727 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.527679 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da"} err="failed to get container status \"fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da\": rpc error: code = NotFound desc = could not find container \"fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da\": container with ID starting with fa324b71f59780cdda2f5553a895728f00621551182a04b29908083f18bb82da not found: ID does not exist" Jul 24 13:36:24.543440 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.543410 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d"] Jul 24 13:36:24.546759 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.546741 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09af388a-8a77-4a22-9694-a1f05fe0af6d-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:36:24.546816 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.546762 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kbng\" (UniqueName: \"kubernetes.io/projected/09af388a-8a77-4a22-9694-a1f05fe0af6d-kube-api-access-5kbng\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:36:24.549798 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:24.549776 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-77101517-7c556bb488-5489d"] Jul 24 13:36:25.197374 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:25.197334 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09af388a-8a77-4a22-9694-a1f05fe0af6d" path="/var/lib/kubelet/pods/09af388a-8a77-4a22-9694-a1f05fe0af6d/volumes" Jul 24 13:36:27.118339 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:27.118297 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:36:28.675590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:28.675554 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:28.796450 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:28.796413 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:29.069582 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:29.069498 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:29.344136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:29.344045 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:29.440592 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:29.440551 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:29.571429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:29.571371 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:29.961430 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:29.961377 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:30.091470 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:30.091428 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:32.251136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:32.251090 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:32.423036 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:32.423001 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:32.490908 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:32.490870 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:32.911685 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:32.911647 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:33.024895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.024857 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:33.119818 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.119786 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:36:33.150529 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.150493 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs"] Jul 24 13:36:33.150926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.150913 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09af388a-8a77-4a22-9694-a1f05fe0af6d" containerName="payload-pre-processing" Jul 24 13:36:33.150971 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.150929 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09af388a-8a77-4a22-9694-a1f05fe0af6d" containerName="payload-pre-processing" Jul 24 13:36:33.151005 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.150996 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="09af388a-8a77-4a22-9694-a1f05fe0af6d" containerName="payload-pre-processing" Jul 24 13:36:33.154036 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.154019 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.156864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.156839 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-51268fa8\"" Jul 24 13:36:33.156978 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.156921 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-51268fa8-dockercfg-nl2fc\"" Jul 24 13:36:33.165242 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.165186 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs"] Jul 24 13:36:33.222418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.222373 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfmbz\" (UniqueName: \"kubernetes.io/projected/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-kube-api-access-zfmbz\") pod \"payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.222584 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.222476 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-plugins-config-volume\") pod \"payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.323459 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.323415 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-plugins-config-volume\") pod \"payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.323851 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.323486 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfmbz\" (UniqueName: \"kubernetes.io/projected/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-kube-api-access-zfmbz\") pod \"payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.324244 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.324205 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-plugins-config-volume\") pod \"payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.332266 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.332236 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfmbz\" (UniqueName: \"kubernetes.io/projected/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-kube-api-access-zfmbz\") pod \"payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.466798 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.466713 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:33.612091 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:33.612066 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs"] Jul 24 13:36:33.613990 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:36:33.613959 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e2b016_a2b6_4b8e_91f9_6e04a0283f32.slice/crio-4702e5b3a342db55ac56dd656ec8e05d45d16f88d8e06d49dccdf715a8f7af7b WatchSource:0}: Error finding container 4702e5b3a342db55ac56dd656ec8e05d45d16f88d8e06d49dccdf715a8f7af7b: Status 404 returned error can't find the container with id 4702e5b3a342db55ac56dd656ec8e05d45d16f88d8e06d49dccdf715a8f7af7b Jul 24 13:36:34.574296 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:34.574251 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" event={"ID":"67e2b016-a2b6-4b8e-91f9-6e04a0283f32","Type":"ContainerStarted","Data":"1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8"} Jul 24 13:36:34.574796 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:34.574374 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" event={"ID":"67e2b016-a2b6-4b8e-91f9-6e04a0283f32","Type":"ContainerStarted","Data":"4702e5b3a342db55ac56dd656ec8e05d45d16f88d8e06d49dccdf715a8f7af7b"} Jul 24 13:36:34.574796 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:34.574438 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:36:34.590211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:34.590150 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" podStartSLOduration=1.5901306050000001 podStartE2EDuration="1.590130605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:36:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:36:34.589661125 +0000 UTC m=+1657.953585216" watchObservedRunningTime="2026-07-24 13:36:34.590130605 +0000 UTC m=+1657.954054697" Jul 24 13:36:38.314369 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:38.314330 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:36:45.582397 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:36:45.582348 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:37:06.396320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:06.396282 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:07.876913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:07.876875 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:07.907713 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:07.907677 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs"] Jul 24 13:37:07.907950 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:07.907922 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" podUID="67e2b016-a2b6-4b8e-91f9-6e04a0283f32" containerName="payload-processing" containerID="cri-o://1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8" gracePeriod=30 Jul 24 13:37:07.976899 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:07.976863 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:08.077582 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.077542 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:08.175623 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.175594 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:37:08.234054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.234022 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-plugins-config-volume\") pod \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " Jul 24 13:37:08.234197 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.234070 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfmbz\" (UniqueName: \"kubernetes.io/projected/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-kube-api-access-zfmbz\") pod \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\" (UID: \"67e2b016-a2b6-4b8e-91f9-6e04a0283f32\") " Jul 24 13:37:08.234575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.234459 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67e2b016-a2b6-4b8e-91f9-6e04a0283f32" (UID: "67e2b016-a2b6-4b8e-91f9-6e04a0283f32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:37:08.236160 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.236141 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-kube-api-access-zfmbz" (OuterVolumeSpecName: "kube-api-access-zfmbz") pod "67e2b016-a2b6-4b8e-91f9-6e04a0283f32" (UID: "67e2b016-a2b6-4b8e-91f9-6e04a0283f32"). InnerVolumeSpecName "kube-api-access-zfmbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:37:08.335030 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.335003 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:37:08.335030 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.335030 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfmbz\" (UniqueName: \"kubernetes.io/projected/67e2b016-a2b6-4b8e-91f9-6e04a0283f32-kube-api-access-zfmbz\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:37:08.750369 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.750329 2558 generic.go:356] "Generic (PLEG): container finished" podID="67e2b016-a2b6-4b8e-91f9-6e04a0283f32" containerID="1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8" exitCode=0 Jul 24 13:37:08.750606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.750381 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" Jul 24 13:37:08.750606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.750427 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" event={"ID":"67e2b016-a2b6-4b8e-91f9-6e04a0283f32","Type":"ContainerDied","Data":"1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8"} Jul 24 13:37:08.750606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.750493 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs" event={"ID":"67e2b016-a2b6-4b8e-91f9-6e04a0283f32","Type":"ContainerDied","Data":"4702e5b3a342db55ac56dd656ec8e05d45d16f88d8e06d49dccdf715a8f7af7b"} Jul 24 13:37:08.750606 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.750545 2558 scope.go:117] "RemoveContainer" containerID="1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8" Jul 24 13:37:08.759806 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.759789 2558 scope.go:117] "RemoveContainer" containerID="1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8" Jul 24 13:37:08.760073 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:37:08.760051 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8\": container with ID starting with 1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8 not found: ID does not exist" containerID="1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8" Jul 24 13:37:08.760146 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.760086 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8"} err="failed to get container status \"1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8\": rpc error: code = NotFound desc = could not find container \"1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8\": container with ID starting with 1e91f0aa8114438fb426de53dd3f5a690578fa653e27a543db8a0bc6ea24eaa8 not found: ID does not exist" Jul 24 13:37:08.773872 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.773851 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs"] Jul 24 13:37:08.777590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:08.777569 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-51268fa8-54f8bf4ff7-n6sbs"] Jul 24 13:37:09.197654 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:09.197619 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e2b016-a2b6-4b8e-91f9-6e04a0283f32" path="/var/lib/kubelet/pods/67e2b016-a2b6-4b8e-91f9-6e04a0283f32/volumes" Jul 24 13:37:10.861199 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:10.861160 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:37:12.395669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:12.395636 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:12.560593 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:12.560556 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:12.869875 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:12.869840 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:13.064123 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:13.064089 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:13.398448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:13.398411 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:13.705211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:13.705123 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:13.937029 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:13.936992 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:15.312265 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:15.312229 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:15.417184 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:15.417150 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:15.505800 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:15.505769 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:15.719825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:15.719792 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:16.109021 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.108988 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:16.225406 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.225353 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr"] Jul 24 13:37:16.225764 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.225750 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e2b016-a2b6-4b8e-91f9-6e04a0283f32" containerName="payload-processing" Jul 24 13:37:16.225809 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.225766 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e2b016-a2b6-4b8e-91f9-6e04a0283f32" containerName="payload-processing" Jul 24 13:37:16.225847 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.225830 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e2b016-a2b6-4b8e-91f9-6e04a0283f32" containerName="payload-processing" Jul 24 13:37:16.230227 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.230202 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.230550 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.230523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:16.232689 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.232667 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ca810d9a\"" Jul 24 13:37:16.240939 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.240915 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn"] Jul 24 13:37:16.249632 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.249600 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr"] Jul 24 13:37:16.249748 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.249691 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.252256 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.252235 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ca810d9a-dockercfg-2zdz9\"" Jul 24 13:37:16.255891 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.255860 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn"] Jul 24 13:37:16.306179 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.306140 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2344f3ca-4dec-4875-b9ab-c234a65b8f05-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.306482 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.306195 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmr6f\" (UniqueName: \"kubernetes.io/projected/2344f3ca-4dec-4875-b9ab-c234a65b8f05-kube-api-access-wmr6f\") pod \"payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.306482 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.306230 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrq9m\" (UniqueName: \"kubernetes.io/projected/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-kube-api-access-wrq9m\") pod \"payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.306482 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.306290 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-plugins-config-volume\") pod \"payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.407508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.407381 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-plugins-config-volume\") pod \"payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.407508 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.407498 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2344f3ca-4dec-4875-b9ab-c234a65b8f05-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.407981 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.407542 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmr6f\" (UniqueName: \"kubernetes.io/projected/2344f3ca-4dec-4875-b9ab-c234a65b8f05-kube-api-access-wmr6f\") pod \"payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.407981 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.407567 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrq9m\" (UniqueName: \"kubernetes.io/projected/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-kube-api-access-wrq9m\") pod \"payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.408155 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.408135 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-plugins-config-volume\") pod \"payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.408275 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.408257 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2344f3ca-4dec-4875-b9ab-c234a65b8f05-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.421270 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.421236 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmr6f\" (UniqueName: \"kubernetes.io/projected/2344f3ca-4dec-4875-b9ab-c234a65b8f05-kube-api-access-wmr6f\") pod \"payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.421409 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.421322 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrq9m\" (UniqueName: \"kubernetes.io/projected/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-kube-api-access-wrq9m\") pod \"payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.542586 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.542553 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.563646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.563593 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:16.705150 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.704947 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr"] Jul 24 13:37:16.707585 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:37:16.707557 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2344f3ca_4dec_4875_b9ab_c234a65b8f05.slice/crio-73a79a940080d0085431cd75ae6b6f877d940c9229b307fdc55220e647d850b0 WatchSource:0}: Error finding container 73a79a940080d0085431cd75ae6b6f877d940c9229b307fdc55220e647d850b0: Status 404 returned error can't find the container with id 73a79a940080d0085431cd75ae6b6f877d940c9229b307fdc55220e647d850b0 Jul 24 13:37:16.793281 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.793248 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" event={"ID":"2344f3ca-4dec-4875-b9ab-c234a65b8f05","Type":"ContainerStarted","Data":"fddd364a7fa31d7672e9582b305423d9cc075b51c9814a5b643d2ceb41cad241"} Jul 24 13:37:16.793448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.793300 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" event={"ID":"2344f3ca-4dec-4875-b9ab-c234a65b8f05","Type":"ContainerStarted","Data":"73a79a940080d0085431cd75ae6b6f877d940c9229b307fdc55220e647d850b0"} Jul 24 13:37:16.793448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.793348 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:16.812429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.812217 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" podStartSLOduration=0.812198389 podStartE2EDuration="812.198389ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:37:16.811764452 +0000 UTC m=+1700.175688543" watchObservedRunningTime="2026-07-24 13:37:16.812198389 +0000 UTC m=+1700.176122459" Jul 24 13:37:16.930289 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:16.930264 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn"] Jul 24 13:37:16.932050 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:37:16.932021 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d17488_9925_4c5d_a06b_bc13ddb9c7a9.slice/crio-da12135bd81f8778053115daf2d070ec093190467e349e61f06f249d3f3710bc WatchSource:0}: Error finding container da12135bd81f8778053115daf2d070ec093190467e349e61f06f249d3f3710bc: Status 404 returned error can't find the container with id da12135bd81f8778053115daf2d070ec093190467e349e61f06f249d3f3710bc Jul 24 13:37:17.800241 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:17.800200 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" event={"ID":"16d17488-9925-4c5d-a06b-bc13ddb9c7a9","Type":"ContainerStarted","Data":"6540b6c299e5e81d72c66fb5e976b96f594897f0b40e6e11d0be3da33ac04b2c"} Jul 24 13:37:17.800689 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:17.800258 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" event={"ID":"16d17488-9925-4c5d-a06b-bc13ddb9c7a9","Type":"ContainerStarted","Data":"da12135bd81f8778053115daf2d070ec093190467e349e61f06f249d3f3710bc"} Jul 24 13:37:17.826935 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:17.826884 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" podStartSLOduration=1.826868836 podStartE2EDuration="1.826868836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:37:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:37:17.824987869 +0000 UTC m=+1701.188911960" watchObservedRunningTime="2026-07-24 13:37:17.826868836 +0000 UTC m=+1701.190792926" Jul 24 13:37:18.805478 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:18.805435 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:22.618699 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:22.618364 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:37:27.802323 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:27.802293 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:29.813219 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:29.813182 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:39.829125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:39.829091 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:39.930733 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:39.930694 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:51.732782 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.732743 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:51.763802 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.763766 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn"] Jul 24 13:37:51.764053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.764020 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" podUID="16d17488-9925-4c5d-a06b-bc13ddb9c7a9" containerName="payload-processing" containerID="cri-o://6540b6c299e5e81d72c66fb5e976b96f594897f0b40e6e11d0be3da33ac04b2c" gracePeriod=30 Jul 24 13:37:51.768076 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.767967 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr"] Jul 24 13:37:51.768805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.768740 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" podUID="2344f3ca-4dec-4875-b9ab-c234a65b8f05" containerName="payload-pre-processing" containerID="cri-o://fddd364a7fa31d7672e9582b305423d9cc075b51c9814a5b643d2ceb41cad241" gracePeriod=30 Jul 24 13:37:51.803063 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.803028 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:51.872271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.872232 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:51.970361 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.970222 2558 generic.go:356] "Generic (PLEG): container finished" podID="2344f3ca-4dec-4875-b9ab-c234a65b8f05" containerID="fddd364a7fa31d7672e9582b305423d9cc075b51c9814a5b643d2ceb41cad241" exitCode=0 Jul 24 13:37:51.970361 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.970295 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" event={"ID":"2344f3ca-4dec-4875-b9ab-c234a65b8f05","Type":"ContainerDied","Data":"fddd364a7fa31d7672e9582b305423d9cc075b51c9814a5b643d2ceb41cad241"} Jul 24 13:37:51.973293 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.972376 2558 generic.go:356] "Generic (PLEG): container finished" podID="16d17488-9925-4c5d-a06b-bc13ddb9c7a9" containerID="6540b6c299e5e81d72c66fb5e976b96f594897f0b40e6e11d0be3da33ac04b2c" exitCode=0 Jul 24 13:37:51.973293 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:51.972444 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" event={"ID":"16d17488-9925-4c5d-a06b-bc13ddb9c7a9","Type":"ContainerDied","Data":"6540b6c299e5e81d72c66fb5e976b96f594897f0b40e6e11d0be3da33ac04b2c"} Jul 24 13:37:52.056563 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.056538 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:52.059855 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.059827 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:52.123051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.123023 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmr6f\" (UniqueName: \"kubernetes.io/projected/2344f3ca-4dec-4875-b9ab-c234a65b8f05-kube-api-access-wmr6f\") pod \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " Jul 24 13:37:52.123051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.123055 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-plugins-config-volume\") pod \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " Jul 24 13:37:52.123255 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.123076 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2344f3ca-4dec-4875-b9ab-c234a65b8f05-plugins-config-volume\") pod \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\" (UID: \"2344f3ca-4dec-4875-b9ab-c234a65b8f05\") " Jul 24 13:37:52.123255 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.123134 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrq9m\" (UniqueName: \"kubernetes.io/projected/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-kube-api-access-wrq9m\") pod \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\" (UID: \"16d17488-9925-4c5d-a06b-bc13ddb9c7a9\") " Jul 24 13:37:52.123553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.123522 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2344f3ca-4dec-4875-b9ab-c234a65b8f05-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2344f3ca-4dec-4875-b9ab-c234a65b8f05" (UID: "2344f3ca-4dec-4875-b9ab-c234a65b8f05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:37:52.123672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.123522 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16d17488-9925-4c5d-a06b-bc13ddb9c7a9" (UID: "16d17488-9925-4c5d-a06b-bc13ddb9c7a9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:37:52.125260 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.125233 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-kube-api-access-wrq9m" (OuterVolumeSpecName: "kube-api-access-wrq9m") pod "16d17488-9925-4c5d-a06b-bc13ddb9c7a9" (UID: "16d17488-9925-4c5d-a06b-bc13ddb9c7a9"). InnerVolumeSpecName "kube-api-access-wrq9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:37:52.125350 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.125257 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2344f3ca-4dec-4875-b9ab-c234a65b8f05-kube-api-access-wmr6f" (OuterVolumeSpecName: "kube-api-access-wmr6f") pod "2344f3ca-4dec-4875-b9ab-c234a65b8f05" (UID: "2344f3ca-4dec-4875-b9ab-c234a65b8f05"). InnerVolumeSpecName "kube-api-access-wmr6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:37:52.224716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.224684 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmr6f\" (UniqueName: \"kubernetes.io/projected/2344f3ca-4dec-4875-b9ab-c234a65b8f05-kube-api-access-wmr6f\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:37:52.224716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.224709 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:37:52.224716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.224719 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2344f3ca-4dec-4875-b9ab-c234a65b8f05-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:37:52.224716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.224728 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrq9m\" (UniqueName: \"kubernetes.io/projected/16d17488-9925-4c5d-a06b-bc13ddb9c7a9-kube-api-access-wrq9m\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:37:52.979144 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.979114 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" Jul 24 13:37:52.979602 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.979155 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr" event={"ID":"2344f3ca-4dec-4875-b9ab-c234a65b8f05","Type":"ContainerDied","Data":"73a79a940080d0085431cd75ae6b6f877d940c9229b307fdc55220e647d850b0"} Jul 24 13:37:52.979602 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.979220 2558 scope.go:117] "RemoveContainer" containerID="fddd364a7fa31d7672e9582b305423d9cc075b51c9814a5b643d2ceb41cad241" Jul 24 13:37:52.980688 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.980664 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" event={"ID":"16d17488-9925-4c5d-a06b-bc13ddb9c7a9","Type":"ContainerDied","Data":"da12135bd81f8778053115daf2d070ec093190467e349e61f06f249d3f3710bc"} Jul 24 13:37:52.980786 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.980694 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn" Jul 24 13:37:52.988714 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:52.988577 2558 scope.go:117] "RemoveContainer" containerID="6540b6c299e5e81d72c66fb5e976b96f594897f0b40e6e11d0be3da33ac04b2c" Jul 24 13:37:53.004126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:53.004104 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr"] Jul 24 13:37:53.009976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:53.009953 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ca810d9a-654db4c6b4-6t7mr"] Jul 24 13:37:53.021661 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:53.021638 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn"] Jul 24 13:37:53.027539 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:53.027514 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ca810d9a-586fbc5c98-vvxkn"] Jul 24 13:37:53.196821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:53.196788 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d17488-9925-4c5d-a06b-bc13ddb9c7a9" path="/var/lib/kubelet/pods/16d17488-9925-4c5d-a06b-bc13ddb9c7a9/volumes" Jul 24 13:37:53.197159 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:53.197147 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2344f3ca-4dec-4875-b9ab-c234a65b8f05" path="/var/lib/kubelet/pods/2344f3ca-4dec-4875-b9ab-c234a65b8f05/volumes" Jul 24 13:37:54.916400 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:54.916354 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:37:56.324160 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:56.324105 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:57.608226 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:57.608196 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:57.859898 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:57.859793 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:58.108134 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:58.108100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:58.448378 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:58.448343 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:37:58.851355 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:37:58.851317 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:00.661150 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:00.661114 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:00.761664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:00.761625 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:00.872925 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:00.872889 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:01.150838 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.150801 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:01.460296 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.460206 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:01.555658 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.555623 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:01.577033 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577007 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d"] Jul 24 13:38:01.577496 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577474 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2344f3ca-4dec-4875-b9ab-c234a65b8f05" containerName="payload-pre-processing" Jul 24 13:38:01.577640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577612 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2344f3ca-4dec-4875-b9ab-c234a65b8f05" containerName="payload-pre-processing" Jul 24 13:38:01.577715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577676 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d17488-9925-4c5d-a06b-bc13ddb9c7a9" containerName="payload-processing" Jul 24 13:38:01.577715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577688 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d17488-9925-4c5d-a06b-bc13ddb9c7a9" containerName="payload-processing" Jul 24 13:38:01.577858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577843 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2344f3ca-4dec-4875-b9ab-c234a65b8f05" containerName="payload-pre-processing" Jul 24 13:38:01.577924 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.577865 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d17488-9925-4c5d-a06b-bc13ddb9c7a9" containerName="payload-processing" Jul 24 13:38:01.580704 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.580684 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.583429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.583407 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-a35f265b\"" Jul 24 13:38:01.583547 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.583411 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-a35f265b-dockercfg-7w6jb\"" Jul 24 13:38:01.590588 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.590564 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d"] Jul 24 13:38:01.700948 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.700914 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m6rw\" (UniqueName: \"kubernetes.io/projected/dda767a9-3ac6-4650-b869-991759140a7c-kube-api-access-2m6rw\") pod \"payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.701289 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.700980 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda767a9-3ac6-4650-b869-991759140a7c-plugins-config-volume\") pod \"payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.801903 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.801837 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m6rw\" (UniqueName: \"kubernetes.io/projected/dda767a9-3ac6-4650-b869-991759140a7c-kube-api-access-2m6rw\") pod \"payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.801903 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.801886 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda767a9-3ac6-4650-b869-991759140a7c-plugins-config-volume\") pod \"payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.802548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.802530 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda767a9-3ac6-4650-b869-991759140a7c-plugins-config-volume\") pod \"payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.811492 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.811469 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m6rw\" (UniqueName: \"kubernetes.io/projected/dda767a9-3ac6-4650-b869-991759140a7c-kube-api-access-2m6rw\") pod \"payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:01.892368 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:01.892342 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:02.025664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:02.025642 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d"] Jul 24 13:38:02.027307 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:38:02.027269 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda767a9_3ac6_4650_b869_991759140a7c.slice/crio-9e2b5b1dd26abca6b755564cc641e5fecc69ffbb0093bc4fe1235df31286c9e5 WatchSource:0}: Error finding container 9e2b5b1dd26abca6b755564cc641e5fecc69ffbb0093bc4fe1235df31286c9e5: Status 404 returned error can't find the container with id 9e2b5b1dd26abca6b755564cc641e5fecc69ffbb0093bc4fe1235df31286c9e5 Jul 24 13:38:02.040313 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:02.040287 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" event={"ID":"dda767a9-3ac6-4650-b869-991759140a7c","Type":"ContainerStarted","Data":"9e2b5b1dd26abca6b755564cc641e5fecc69ffbb0093bc4fe1235df31286c9e5"} Jul 24 13:38:03.047567 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:03.047532 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" event={"ID":"dda767a9-3ac6-4650-b869-991759140a7c","Type":"ContainerStarted","Data":"c95b4adf21ec14b7de82d63927c89e1524801889572c7e8cb2768847be22b38f"} Jul 24 13:38:03.047951 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:03.047657 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:03.066437 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:03.066380 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" podStartSLOduration=2.066368158 podStartE2EDuration="2.066368158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:38:03.064337738 +0000 UTC m=+1746.428261829" watchObservedRunningTime="2026-07-24 13:38:03.066368158 +0000 UTC m=+1746.430292248" Jul 24 13:38:06.114636 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:06.114594 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:38:14.056105 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:14.056070 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:35.983903 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:35.983845 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:36.010414 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.010372 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d"] Jul 24 13:38:36.010646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.010623 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" podUID="dda767a9-3ac6-4650-b869-991759140a7c" containerName="payload-processing" containerID="cri-o://c95b4adf21ec14b7de82d63927c89e1524801889572c7e8cb2768847be22b38f" gracePeriod=30 Jul 24 13:38:36.080871 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.080835 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:36.181973 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.181912 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:36.221302 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.221276 2558 generic.go:356] "Generic (PLEG): container finished" podID="dda767a9-3ac6-4650-b869-991759140a7c" containerID="c95b4adf21ec14b7de82d63927c89e1524801889572c7e8cb2768847be22b38f" exitCode=0 Jul 24 13:38:36.221479 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.221328 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" event={"ID":"dda767a9-3ac6-4650-b869-991759140a7c","Type":"ContainerDied","Data":"c95b4adf21ec14b7de82d63927c89e1524801889572c7e8cb2768847be22b38f"} Jul 24 13:38:36.276343 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.276314 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:36.380055 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.380023 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda767a9-3ac6-4650-b869-991759140a7c-plugins-config-volume\") pod \"dda767a9-3ac6-4650-b869-991759140a7c\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " Jul 24 13:38:36.380231 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.380137 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m6rw\" (UniqueName: \"kubernetes.io/projected/dda767a9-3ac6-4650-b869-991759140a7c-kube-api-access-2m6rw\") pod \"dda767a9-3ac6-4650-b869-991759140a7c\" (UID: \"dda767a9-3ac6-4650-b869-991759140a7c\") " Jul 24 13:38:36.380416 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.380369 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda767a9-3ac6-4650-b869-991759140a7c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dda767a9-3ac6-4650-b869-991759140a7c" (UID: "dda767a9-3ac6-4650-b869-991759140a7c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:38:36.380505 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.380480 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dda767a9-3ac6-4650-b869-991759140a7c-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:38:36.382194 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.382172 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda767a9-3ac6-4650-b869-991759140a7c-kube-api-access-2m6rw" (OuterVolumeSpecName: "kube-api-access-2m6rw") pod "dda767a9-3ac6-4650-b869-991759140a7c" (UID: "dda767a9-3ac6-4650-b869-991759140a7c"). InnerVolumeSpecName "kube-api-access-2m6rw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:38:36.481842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:36.481810 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m6rw\" (UniqueName: \"kubernetes.io/projected/dda767a9-3ac6-4650-b869-991759140a7c-kube-api-access-2m6rw\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:38:37.226922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:37.226898 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" Jul 24 13:38:37.227265 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:37.226900 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d" event={"ID":"dda767a9-3ac6-4650-b869-991759140a7c","Type":"ContainerDied","Data":"9e2b5b1dd26abca6b755564cc641e5fecc69ffbb0093bc4fe1235df31286c9e5"} Jul 24 13:38:37.227265 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:37.227031 2558 scope.go:117] "RemoveContainer" containerID="c95b4adf21ec14b7de82d63927c89e1524801889572c7e8cb2768847be22b38f" Jul 24 13:38:37.244090 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:37.244065 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d"] Jul 24 13:38:37.247460 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:37.247441 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-a35f265b-586bdb59f8-ltj7d"] Jul 24 13:38:38.611922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:38.611886 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:38:39.196225 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:39.196182 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dda767a9-3ac6-4650-b869-991759140a7c" path="/var/lib/kubelet/pods/dda767a9-3ac6-4650-b869-991759140a7c/volumes" Jul 24 13:38:40.582114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:40.582080 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:40.754298 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:40.754264 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:40.991187 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:40.991148 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:41.507733 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:41.507693 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:41.608184 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:41.608156 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:41.707031 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:41.706961 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:43.227775 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:43.227744 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:49.035473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:49.035419 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:49.137021 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:49.136984 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:49.240330 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:49.240288 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:49.336331 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:49.336226 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:49.959984 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:49.959948 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:57.188922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:57.188895 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:38:57.199917 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:57.199895 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:38:57.241878 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:57.241850 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:38:57.343580 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:38:57.343550 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:15.821884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:15.821848 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:22.122145 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:22.122112 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:22.231258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:22.231228 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:22.322512 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:22.322481 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:38.324332 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:38.324293 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:44.143349 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:44.143314 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:44.325842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:44.325810 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:44.424583 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:44.424516 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:54.907545 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:54.907507 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:39:55.748186 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:39:55.748159 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:02.409421 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:02.409365 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:02.506942 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:02.506909 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:02.609800 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:02.609772 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.399970 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:03.399924 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.609779 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:03.609743 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:03.703684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:03.703612 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:05.285725 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:05.285693 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:05.817211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:05.817179 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:40:11.778808 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:11.778767 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:11.878680 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:11.878655 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:12.480444 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:12.480411 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:15.116033 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:15.116000 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:40:18.611208 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:18.611181 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:19.808793 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:19.808713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:22.161366 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:22.161328 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:22.255744 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:22.255715 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:22.355238 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:22.355203 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:22.874445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:22.874406 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.046961 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.046930 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.153330 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.153246 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:23.164581 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.164556 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4"] Jul 24 13:40:23.164964 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.164951 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dda767a9-3ac6-4650-b869-991759140a7c" containerName="payload-processing" Jul 24 13:40:23.164964 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.164965 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="dda767a9-3ac6-4650-b869-991759140a7c" containerName="payload-processing" Jul 24 13:40:23.165038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.165031 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="dda767a9-3ac6-4650-b869-991759140a7c" containerName="payload-processing" Jul 24 13:40:23.168168 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.168151 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.170617 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.170592 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bcb704f0-dockercfg-7wswd\"" Jul 24 13:40:23.170728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.170594 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bcb704f0\"" Jul 24 13:40:23.176674 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.176572 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4"] Jul 24 13:40:23.280403 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.280357 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44370b71-a902-4cc2-b04f-bf7223ec32c7-plugins-config-volume\") pod \"payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.280515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.280438 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wgq\" (UniqueName: \"kubernetes.io/projected/44370b71-a902-4cc2-b04f-bf7223ec32c7-kube-api-access-79wgq\") pod \"payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.381051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.381027 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44370b71-a902-4cc2-b04f-bf7223ec32c7-plugins-config-volume\") pod \"payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.381181 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.381061 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wgq\" (UniqueName: \"kubernetes.io/projected/44370b71-a902-4cc2-b04f-bf7223ec32c7-kube-api-access-79wgq\") pod \"payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.381685 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.381666 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44370b71-a902-4cc2-b04f-bf7223ec32c7-plugins-config-volume\") pod \"payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.389648 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.389630 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wgq\" (UniqueName: \"kubernetes.io/projected/44370b71-a902-4cc2-b04f-bf7223ec32c7-kube-api-access-79wgq\") pod \"payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.479957 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.479898 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.616865 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.616805 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4"] Jul 24 13:40:23.619203 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:40:23.619173 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44370b71_a902_4cc2_b04f_bf7223ec32c7.slice/crio-507e99f0019a5f32cc5538e94a611205a9f06a2f9ab170d2f96547eb04391ebe WatchSource:0}: Error finding container 507e99f0019a5f32cc5538e94a611205a9f06a2f9ab170d2f96547eb04391ebe: Status 404 returned error can't find the container with id 507e99f0019a5f32cc5538e94a611205a9f06a2f9ab170d2f96547eb04391ebe Jul 24 13:40:23.749206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.749149 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" event={"ID":"44370b71-a902-4cc2-b04f-bf7223ec32c7","Type":"ContainerStarted","Data":"8e9f4a37116ba3f524a30f42e8a422c6950d275e10f0bf5787541659f5023c0c"} Jul 24 13:40:23.749206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.749195 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" event={"ID":"44370b71-a902-4cc2-b04f-bf7223ec32c7","Type":"ContainerStarted","Data":"507e99f0019a5f32cc5538e94a611205a9f06a2f9ab170d2f96547eb04391ebe"} Jul 24 13:40:23.749363 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.749307 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:23.768810 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:23.768757 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" podStartSLOduration=0.768740033 podStartE2EDuration="768.740033ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:23.765628147 +0000 UTC m=+1887.129552237" watchObservedRunningTime="2026-07-24 13:40:23.768740033 +0000 UTC m=+1887.132664124" Jul 24 13:40:24.919838 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:24.919806 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:40:26.685376 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:26.685343 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:27.395860 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:27.395828 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:27.489808 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:27.489782 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:27.850940 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:27.850908 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:33.784568 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:33.784535 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:33.911696 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:33.911669 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:34.755874 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:34.755845 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:44.676438 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.676396 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:44.702221 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.702194 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4"] Jul 24 13:40:44.702515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.702484 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" podUID="44370b71-a902-4cc2-b04f-bf7223ec32c7" containerName="payload-processing" containerID="cri-o://8e9f4a37116ba3f524a30f42e8a422c6950d275e10f0bf5787541659f5023c0c" gracePeriod=30 Jul 24 13:40:44.754131 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.754099 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" podUID="44370b71-a902-4cc2-b04f-bf7223ec32c7" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.53:9004: connect: connection refused" Jul 24 13:40:44.776782 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.776755 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:44.852327 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.852300 2558 generic.go:356] "Generic (PLEG): container finished" podID="44370b71-a902-4cc2-b04f-bf7223ec32c7" containerID="8e9f4a37116ba3f524a30f42e8a422c6950d275e10f0bf5787541659f5023c0c" exitCode=0 Jul 24 13:40:44.852443 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.852367 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" event={"ID":"44370b71-a902-4cc2-b04f-bf7223ec32c7","Type":"ContainerDied","Data":"8e9f4a37116ba3f524a30f42e8a422c6950d275e10f0bf5787541659f5023c0c"} Jul 24 13:40:44.880126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.880075 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:44.952037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:44.952010 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:45.046494 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.046471 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44370b71-a902-4cc2-b04f-bf7223ec32c7-plugins-config-volume\") pod \"44370b71-a902-4cc2-b04f-bf7223ec32c7\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " Jul 24 13:40:45.046621 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.046519 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79wgq\" (UniqueName: \"kubernetes.io/projected/44370b71-a902-4cc2-b04f-bf7223ec32c7-kube-api-access-79wgq\") pod \"44370b71-a902-4cc2-b04f-bf7223ec32c7\" (UID: \"44370b71-a902-4cc2-b04f-bf7223ec32c7\") " Jul 24 13:40:45.046845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.046822 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44370b71-a902-4cc2-b04f-bf7223ec32c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44370b71-a902-4cc2-b04f-bf7223ec32c7" (UID: "44370b71-a902-4cc2-b04f-bf7223ec32c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:40:45.048426 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.048406 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44370b71-a902-4cc2-b04f-bf7223ec32c7-kube-api-access-79wgq" (OuterVolumeSpecName: "kube-api-access-79wgq") pod "44370b71-a902-4cc2-b04f-bf7223ec32c7" (UID: "44370b71-a902-4cc2-b04f-bf7223ec32c7"). InnerVolumeSpecName "kube-api-access-79wgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:40:45.147730 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.147708 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44370b71-a902-4cc2-b04f-bf7223ec32c7-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:40:45.147730 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.147730 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79wgq\" (UniqueName: \"kubernetes.io/projected/44370b71-a902-4cc2-b04f-bf7223ec32c7-kube-api-access-79wgq\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:40:45.858200 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.858173 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" Jul 24 13:40:45.858640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.858202 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4" event={"ID":"44370b71-a902-4cc2-b04f-bf7223ec32c7","Type":"ContainerDied","Data":"507e99f0019a5f32cc5538e94a611205a9f06a2f9ab170d2f96547eb04391ebe"} Jul 24 13:40:45.858640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.858267 2558 scope.go:117] "RemoveContainer" containerID="8e9f4a37116ba3f524a30f42e8a422c6950d275e10f0bf5787541659f5023c0c" Jul 24 13:40:45.874736 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.874710 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4"] Jul 24 13:40:45.879036 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:45.879015 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bcb704f0-6fdc6c454-22tb4"] Jul 24 13:40:47.196941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:47.196911 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44370b71-a902-4cc2-b04f-bf7223ec32c7" path="/var/lib/kubelet/pods/44370b71-a902-4cc2-b04f-bf7223ec32c7/volumes" Jul 24 13:40:47.513253 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:47.513175 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:40:49.228877 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:49.228840 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:49.704023 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:49.703993 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:55.646914 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:55.646867 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:55.747687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:55.747649 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:55.960337 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:55.960248 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:56.256498 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:56.256419 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:56.434465 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:56.434437 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:56.532468 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:56.532381 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:40:57.316128 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:40:57.316100 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:02.458613 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:02.458584 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:02.519395 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:02.519349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:02.658716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:02.658688 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:02.806550 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:02.806492 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:02.901170 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:02.901150 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:08.599727 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:08.599687 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:09.248742 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:09.248707 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:15.829713 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:15.829675 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:16.263348 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:16.259887 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:16.545521 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:16.545445 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:19.207355 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:19.207322 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:19.304301 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:19.304266 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:19.375168 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:19.375144 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:19.559101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:19.559030 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:19.890445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:19.890409 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:19.990957 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:19.990922 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:20.026984 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.026955 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z"] Jul 24 13:41:20.027363 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.027350 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44370b71-a902-4cc2-b04f-bf7223ec32c7" containerName="payload-processing" Jul 24 13:41:20.027434 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.027364 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="44370b71-a902-4cc2-b04f-bf7223ec32c7" containerName="payload-processing" Jul 24 13:41:20.027476 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.027469 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="44370b71-a902-4cc2-b04f-bf7223ec32c7" containerName="payload-processing" Jul 24 13:41:20.030630 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.030610 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.033082 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.033064 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2cdea471\"" Jul 24 13:41:20.033187 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.033098 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2cdea471-dockercfg-9z8zr\"" Jul 24 13:41:20.039505 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.039473 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z"] Jul 24 13:41:20.207137 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.207069 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/752bbc74-8104-4032-bffe-a560c98f4c32-plugins-config-volume\") pod \"payload-processing-e2e-mt-2cdea471-94476b98b-klg6z\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.207137 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.207125 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlpm\" (UniqueName: \"kubernetes.io/projected/752bbc74-8104-4032-bffe-a560c98f4c32-kube-api-access-jmlpm\") pod \"payload-processing-e2e-mt-2cdea471-94476b98b-klg6z\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.308484 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.308453 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlpm\" (UniqueName: \"kubernetes.io/projected/752bbc74-8104-4032-bffe-a560c98f4c32-kube-api-access-jmlpm\") pod \"payload-processing-e2e-mt-2cdea471-94476b98b-klg6z\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.308823 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.308688 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/752bbc74-8104-4032-bffe-a560c98f4c32-plugins-config-volume\") pod \"payload-processing-e2e-mt-2cdea471-94476b98b-klg6z\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.309295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.309275 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/752bbc74-8104-4032-bffe-a560c98f4c32-plugins-config-volume\") pod \"payload-processing-e2e-mt-2cdea471-94476b98b-klg6z\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.317699 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.317676 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlpm\" (UniqueName: \"kubernetes.io/projected/752bbc74-8104-4032-bffe-a560c98f4c32-kube-api-access-jmlpm\") pod \"payload-processing-e2e-mt-2cdea471-94476b98b-klg6z\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.341645 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.341615 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:20.679821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:20.679796 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z"] Jul 24 13:41:20.681745 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:41:20.681717 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752bbc74_8104_4032_bffe_a560c98f4c32.slice/crio-91def861b03939564be75c70479588e86c908cf77bbcd6adf7251dd704378c7f WatchSource:0}: Error finding container 91def861b03939564be75c70479588e86c908cf77bbcd6adf7251dd704378c7f: Status 404 returned error can't find the container with id 91def861b03939564be75c70479588e86c908cf77bbcd6adf7251dd704378c7f Jul 24 13:41:21.033191 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:21.033116 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" event={"ID":"752bbc74-8104-4032-bffe-a560c98f4c32","Type":"ContainerStarted","Data":"f3a4e525a9123a472a1a870de3ef021df17fdccd88280795037c82cd402aedb7"} Jul 24 13:41:21.033191 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:21.033163 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" event={"ID":"752bbc74-8104-4032-bffe-a560c98f4c32","Type":"ContainerStarted","Data":"91def861b03939564be75c70479588e86c908cf77bbcd6adf7251dd704378c7f"} Jul 24 13:41:21.033375 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:21.033232 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:21.049206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:21.049157 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" podStartSLOduration=1.0491425 podStartE2EDuration="1.0491425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:41:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:41:21.048433602 +0000 UTC m=+1944.412357692" watchObservedRunningTime="2026-07-24 13:41:21.0491425 +0000 UTC m=+1944.413066590" Jul 24 13:41:24.041792 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:24.041760 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:26.007473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:26.007435 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:41:29.135458 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:29.135413 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:41:32.040285 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:32.040258 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:41:34.309975 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:34.309940 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:41:50.516823 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:50.516742 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:41:53.635585 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:41:53.635546 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:12.630342 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:12.630285 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:42:14.027715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.027673 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:14.036706 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.036668 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z"] Jul 24 13:42:14.036926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.036900 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" podUID="752bbc74-8104-4032-bffe-a560c98f4c32" containerName="payload-processing" containerID="cri-o://f3a4e525a9123a472a1a870de3ef021df17fdccd88280795037c82cd402aedb7" gracePeriod=30 Jul 24 13:42:14.126871 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.126823 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:14.297988 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.297957 2558 generic.go:356] "Generic (PLEG): container finished" podID="752bbc74-8104-4032-bffe-a560c98f4c32" containerID="f3a4e525a9123a472a1a870de3ef021df17fdccd88280795037c82cd402aedb7" exitCode=0 Jul 24 13:42:14.298141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.298011 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" event={"ID":"752bbc74-8104-4032-bffe-a560c98f4c32","Type":"ContainerDied","Data":"f3a4e525a9123a472a1a870de3ef021df17fdccd88280795037c82cd402aedb7"} Jul 24 13:42:14.298141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.298047 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" event={"ID":"752bbc74-8104-4032-bffe-a560c98f4c32","Type":"ContainerDied","Data":"91def861b03939564be75c70479588e86c908cf77bbcd6adf7251dd704378c7f"} Jul 24 13:42:14.298141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.298063 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91def861b03939564be75c70479588e86c908cf77bbcd6adf7251dd704378c7f" Jul 24 13:42:14.310716 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.310687 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:42:14.445115 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.445073 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmlpm\" (UniqueName: \"kubernetes.io/projected/752bbc74-8104-4032-bffe-a560c98f4c32-kube-api-access-jmlpm\") pod \"752bbc74-8104-4032-bffe-a560c98f4c32\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " Jul 24 13:42:14.445115 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.445122 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/752bbc74-8104-4032-bffe-a560c98f4c32-plugins-config-volume\") pod \"752bbc74-8104-4032-bffe-a560c98f4c32\" (UID: \"752bbc74-8104-4032-bffe-a560c98f4c32\") " Jul 24 13:42:14.445553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.445529 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/752bbc74-8104-4032-bffe-a560c98f4c32-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "752bbc74-8104-4032-bffe-a560c98f4c32" (UID: "752bbc74-8104-4032-bffe-a560c98f4c32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:42:14.447260 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.447235 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752bbc74-8104-4032-bffe-a560c98f4c32-kube-api-access-jmlpm" (OuterVolumeSpecName: "kube-api-access-jmlpm") pod "752bbc74-8104-4032-bffe-a560c98f4c32" (UID: "752bbc74-8104-4032-bffe-a560c98f4c32"). InnerVolumeSpecName "kube-api-access-jmlpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:42:14.546713 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.546619 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmlpm\" (UniqueName: \"kubernetes.io/projected/752bbc74-8104-4032-bffe-a560c98f4c32-kube-api-access-jmlpm\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:42:14.546713 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:14.546655 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/752bbc74-8104-4032-bffe-a560c98f4c32-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:42:15.302441 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:15.302404 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z" Jul 24 13:42:15.318791 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:15.318754 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z"] Jul 24 13:42:15.320471 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:15.320445 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2cdea471-94476b98b-klg6z"] Jul 24 13:42:17.196445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:17.196405 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="752bbc74-8104-4032-bffe-a560c98f4c32" path="/var/lib/kubelet/pods/752bbc74-8104-4032-bffe-a560c98f4c32/volumes" Jul 24 13:42:18.577058 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:18.577018 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:21.611090 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:21.611049 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:42:22.038270 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:22.038179 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:42:27.159051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:42:27.159014 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:03.682576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:03.682538 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:04.488799 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:04.488737 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:09.986410 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:09.986362 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:32.942844 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:32.942800 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:33.043799 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:33.043766 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:33.321686 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:33.321571 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:33.564141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:33.564102 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:34.990629 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:34.990590 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:35.094477 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:35.094444 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:35.189767 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:35.189729 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:37.254715 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:37.254679 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:37.347536 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:37.347238 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:37.444494 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:37.444450 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:39.445083 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:39.445038 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:44.318636 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:44.318591 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:43:45.532671 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:45.532636 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:45.633096 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:45.633053 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:46.348913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:46.348877 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:48.810843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:48.810798 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:43:52.207344 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:52.207292 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:52.320475 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:52.320434 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:52.413144 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:52.413107 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:43:57.225847 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:57.225814 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:43:57.234634 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:43:57.234607 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:44:03.009529 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:03.009471 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:03.393455 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:03.393417 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:17.791570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:17.791528 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:18.850905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:18.850844 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:18.911723 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:18.911681 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:24.398816 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:24.398779 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:30.681577 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:30.681523 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:30.792668 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:30.792616 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:31.065164 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:31.065058 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:31.423349 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:31.423301 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:31.538171 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:31.538107 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:34.226163 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:34.226128 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:34.326353 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:34.326310 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:34.484510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:34.484376 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:34.957134 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:34.957089 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:35.059097 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.059062 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:35.074786 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.074749 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t"] Jul 24 13:44:35.075171 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.075158 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="752bbc74-8104-4032-bffe-a560c98f4c32" containerName="payload-processing" Jul 24 13:44:35.075217 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.075173 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="752bbc74-8104-4032-bffe-a560c98f4c32" containerName="payload-processing" Jul 24 13:44:35.075262 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.075252 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="752bbc74-8104-4032-bffe-a560c98f4c32" containerName="payload-processing" Jul 24 13:44:35.078499 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.078475 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.081730 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.081699 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-50b298-dockercfg-hr9xr\"" Jul 24 13:44:35.081905 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.081780 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-50b298\"" Jul 24 13:44:35.091559 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.091518 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t"] Jul 24 13:44:35.099167 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.099133 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n"] Jul 24 13:44:35.103311 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.103277 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.113160 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.113115 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n"] Jul 24 13:44:35.160334 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.160294 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.160536 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.160363 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgljt\" (UniqueName: \"kubernetes.io/projected/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-kube-api-access-pgljt\") pod \"payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.188585 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.188544 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:35.261590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.261492 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.261590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.261560 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2qj9\" (UniqueName: \"kubernetes.io/projected/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-kube-api-access-c2qj9\") pod \"payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.262039 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.261601 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.262039 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.261658 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgljt\" (UniqueName: \"kubernetes.io/projected/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-kube-api-access-pgljt\") pod \"payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.262310 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.262289 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.271611 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.271580 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgljt\" (UniqueName: \"kubernetes.io/projected/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-kube-api-access-pgljt\") pod \"payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.363351 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.363305 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2qj9\" (UniqueName: \"kubernetes.io/projected/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-kube-api-access-c2qj9\") pod \"payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.363593 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.363374 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.364145 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.364119 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.380127 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.380093 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2qj9\" (UniqueName: \"kubernetes.io/projected/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-kube-api-access-c2qj9\") pod \"payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.391077 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.391042 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:35.417469 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.417434 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:35.570612 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.570458 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t"] Jul 24 13:44:35.573611 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:44:35.573539 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0833a6_5b1f_4bf2_9e7b_29f49c6f77a4.slice/crio-72e0b8935033662662a01f89f79588c69addf491cc82c700f5f57e5a0c2e3bfc WatchSource:0}: Error finding container 72e0b8935033662662a01f89f79588c69addf491cc82c700f5f57e5a0c2e3bfc: Status 404 returned error can't find the container with id 72e0b8935033662662a01f89f79588c69addf491cc82c700f5f57e5a0c2e3bfc Jul 24 13:44:35.600623 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:35.600591 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n"] Jul 24 13:44:35.602458 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:44:35.602344 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8a1dc2_2b62_4e09_bc90_2aeae9565700.slice/crio-7017266d6aa959572e0676228a7ec3454eef4992716562e7586c611a875b105f WatchSource:0}: Error finding container 7017266d6aa959572e0676228a7ec3454eef4992716562e7586c611a875b105f: Status 404 returned error can't find the container with id 7017266d6aa959572e0676228a7ec3454eef4992716562e7586c611a875b105f Jul 24 13:44:36.042315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.042270 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" event={"ID":"7f8a1dc2-2b62-4e09-bc90-2aeae9565700","Type":"ContainerStarted","Data":"db929c4d7622590a510e181bcb87ff7ad3c1beaf50c87995c33e7d7045d9c63a"} Jul 24 13:44:36.042579 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.042344 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" event={"ID":"7f8a1dc2-2b62-4e09-bc90-2aeae9565700","Type":"ContainerStarted","Data":"7017266d6aa959572e0676228a7ec3454eef4992716562e7586c611a875b105f"} Jul 24 13:44:36.042579 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.042415 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:36.044010 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.043979 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" event={"ID":"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4","Type":"ContainerStarted","Data":"ee9de1aa24a84a0d13ac8a00564ed86cf618982bdfa5311d4c92f91fa5ed04b7"} Jul 24 13:44:36.044143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.044027 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" event={"ID":"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4","Type":"ContainerStarted","Data":"72e0b8935033662662a01f89f79588c69addf491cc82c700f5f57e5a0c2e3bfc"} Jul 24 13:44:36.044143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.044104 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:44:36.088053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.087980 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" podStartSLOduration=1.087953474 podStartE2EDuration="1.087953474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:36.086050738 +0000 UTC m=+2139.449974828" watchObservedRunningTime="2026-07-24 13:44:36.087953474 +0000 UTC m=+2139.451877607" Jul 24 13:44:36.121125 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:36.121071 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" podStartSLOduration=1.121048813 podStartE2EDuration="1.121048813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:36.12085086 +0000 UTC m=+2139.484774951" watchObservedRunningTime="2026-07-24 13:44:36.121048813 +0000 UTC m=+2139.484972903" Jul 24 13:44:38.735038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:38.734997 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:39.085342 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:39.085239 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:39.422154 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:39.422101 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:40.078786 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:40.078743 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:42.249082 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:42.249048 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:42.341299 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:42.341262 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:42.423664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:42.423629 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:42.619497 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:42.619453 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:44:42.847825 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:42.847789 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:43.032830 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.032732 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:43.132447 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.132375 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:43.163927 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.163890 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z"] Jul 24 13:44:43.167882 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.167850 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.171137 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.171105 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-7918ee\"" Jul 24 13:44:43.171320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.171107 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-7918ee-dockercfg-km497\"" Jul 24 13:44:43.181202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.181170 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z"] Jul 24 13:44:43.346239 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.346128 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-794bp\" (UniqueName: \"kubernetes.io/projected/e9771fde-3567-47e4-8fce-59da1de5e188-kube-api-access-794bp\") pod \"payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.346239 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.346186 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9771fde-3567-47e4-8fce-59da1de5e188-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.447137 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.447092 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-794bp\" (UniqueName: \"kubernetes.io/projected/e9771fde-3567-47e4-8fce-59da1de5e188-kube-api-access-794bp\") pod \"payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.447315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.447147 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9771fde-3567-47e4-8fce-59da1de5e188-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.447929 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.447908 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9771fde-3567-47e4-8fce-59da1de5e188-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.456053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.456022 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-794bp\" (UniqueName: \"kubernetes.io/projected/e9771fde-3567-47e4-8fce-59da1de5e188-kube-api-access-794bp\") pod \"payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.482926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.482887 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:43.647590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:43.647542 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z"] Jul 24 13:44:43.651263 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:44:43.651230 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9771fde_3567_47e4_8fce_59da1de5e188.slice/crio-9c76c7b79051cefc04a872902b0e21fb611a6f9a75d32e5ef23d566ae0dc67f9 WatchSource:0}: Error finding container 9c76c7b79051cefc04a872902b0e21fb611a6f9a75d32e5ef23d566ae0dc67f9: Status 404 returned error can't find the container with id 9c76c7b79051cefc04a872902b0e21fb611a6f9a75d32e5ef23d566ae0dc67f9 Jul 24 13:44:44.090035 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:44.089912 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" event={"ID":"e9771fde-3567-47e4-8fce-59da1de5e188","Type":"ContainerStarted","Data":"24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed"} Jul 24 13:44:44.090035 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:44.089985 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:44.090035 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:44.090009 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" event={"ID":"e9771fde-3567-47e4-8fce-59da1de5e188","Type":"ContainerStarted","Data":"9c76c7b79051cefc04a872902b0e21fb611a6f9a75d32e5ef23d566ae0dc67f9"} Jul 24 13:44:44.124020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:44.123959 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" podStartSLOduration=1.123940748 podStartE2EDuration="1.123940748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:44.123484871 +0000 UTC m=+2147.487408963" watchObservedRunningTime="2026-07-24 13:44:44.123940748 +0000 UTC m=+2147.487865234" Jul 24 13:44:46.786873 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:46.786834 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:44:47.051678 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:47.051566 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused" Jul 24 13:44:47.052081 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:47.052060 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:44:54.733672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:54.733587 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:44:55.099064 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:55.099033 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:44:55.391765 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:55.391663 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused" Jul 24 13:44:57.050101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:44:57.050045 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused" Jul 24 13:45:00.646261 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:00.646215 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:02.814494 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:02.814454 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:45:07.050932 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:07.050879 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused" Jul 24 13:45:16.814656 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:16.814605 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:45:17.050925 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:17.050891 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:45:23.734286 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:23.734246 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:28.244468 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:28.244417 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:29.625163 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:29.625109 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:45:58.319137 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:45:58.319092 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:11.764895 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:11.764831 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:12.721679 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.721638 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:12.847085 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.847042 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7"] Jul 24 13:46:12.850646 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.850623 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:12.853288 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.853256 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j482l\"" Jul 24 13:46:12.853288 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.853279 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 13:46:12.857883 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.857848 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7"] Jul 24 13:46:12.989249 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:12.989145 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/755af00a-d780-437d-a9b9-c1e97468bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7\" (UID: \"755af00a-d780-437d-a9b9-c1e97468bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:13.089629 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:13.089592 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/755af00a-d780-437d-a9b9-c1e97468bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7\" (UID: \"755af00a-d780-437d-a9b9-c1e97468bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:13.090306 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:13.090281 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/755af00a-d780-437d-a9b9-c1e97468bd01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7\" (UID: \"755af00a-d780-437d-a9b9-c1e97468bd01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:13.164259 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:13.164222 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:13.311549 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:13.311512 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7"] Jul 24 13:46:13.314766 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:46:13.314733 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755af00a_d780_437d_a9b9_c1e97468bd01.slice/crio-c44bef4f1c10de34717c3dfc2df1799aa46f5f953b8af07b39be18f13af9c282 WatchSource:0}: Error finding container c44bef4f1c10de34717c3dfc2df1799aa46f5f953b8af07b39be18f13af9c282: Status 404 returned error can't find the container with id c44bef4f1c10de34717c3dfc2df1799aa46f5f953b8af07b39be18f13af9c282 Jul 24 13:46:13.576942 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:13.576910 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" event={"ID":"755af00a-d780-437d-a9b9-c1e97468bd01","Type":"ContainerStarted","Data":"c44bef4f1c10de34717c3dfc2df1799aa46f5f953b8af07b39be18f13af9c282"} Jul 24 13:46:14.583502 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:14.583467 2558 generic.go:356] "Generic (PLEG): container finished" podID="755af00a-d780-437d-a9b9-c1e97468bd01" containerID="487c59bca0e4172e389fb7b4d72b33aefdfdd592b76ba1a4ef35e811d4973edd" exitCode=0 Jul 24 13:46:14.583930 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:14.583565 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" event={"ID":"755af00a-d780-437d-a9b9-c1e97468bd01","Type":"ContainerDied","Data":"487c59bca0e4172e389fb7b4d72b33aefdfdd592b76ba1a4ef35e811d4973edd"} Jul 24 13:46:15.739079 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:15.739045 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:15.918734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:15.918692 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/755af00a-d780-437d-a9b9-c1e97468bd01-maas-api-service-ca\") pod \"755af00a-d780-437d-a9b9-c1e97468bd01\" (UID: \"755af00a-d780-437d-a9b9-c1e97468bd01\") " Jul 24 13:46:15.919084 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:15.919057 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755af00a-d780-437d-a9b9-c1e97468bd01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "755af00a-d780-437d-a9b9-c1e97468bd01" (UID: "755af00a-d780-437d-a9b9-c1e97468bd01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:46:16.019871 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:16.019820 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/755af00a-d780-437d-a9b9-c1e97468bd01-maas-api-service-ca\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:16.596114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:16.596080 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" Jul 24 13:46:16.596114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:16.596109 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7" event={"ID":"755af00a-d780-437d-a9b9-c1e97468bd01","Type":"ContainerDied","Data":"c44bef4f1c10de34717c3dfc2df1799aa46f5f953b8af07b39be18f13af9c282"} Jul 24 13:46:16.596334 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:16.596159 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c44bef4f1c10de34717c3dfc2df1799aa46f5f953b8af07b39be18f13af9c282" Jul 24 13:46:16.814402 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:16.814329 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:46:23.302686 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.302647 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:23.347728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.347692 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n"] Jul 24 13:46:23.347953 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.347916 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" podUID="7f8a1dc2-2b62-4e09-bc90-2aeae9565700" containerName="payload-processing" containerID="cri-o://db929c4d7622590a510e181bcb87ff7ad3c1beaf50c87995c33e7d7045d9c63a" gracePeriod=30 Jul 24 13:46:23.363468 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.363427 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t"] Jul 24 13:46:23.363998 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.363968 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" containerID="cri-o://ee9de1aa24a84a0d13ac8a00564ed86cf618982bdfa5311d4c92f91fa5ed04b7" gracePeriod=30 Jul 24 13:46:23.396987 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.396950 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:23.465621 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.465577 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:23.638521 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.638476 2558 generic.go:356] "Generic (PLEG): container finished" podID="7f8a1dc2-2b62-4e09-bc90-2aeae9565700" containerID="db929c4d7622590a510e181bcb87ff7ad3c1beaf50c87995c33e7d7045d9c63a" exitCode=0 Jul 24 13:46:23.638687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.638547 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" event={"ID":"7f8a1dc2-2b62-4e09-bc90-2aeae9565700","Type":"ContainerDied","Data":"db929c4d7622590a510e181bcb87ff7ad3c1beaf50c87995c33e7d7045d9c63a"} Jul 24 13:46:23.638687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.638620 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" event={"ID":"7f8a1dc2-2b62-4e09-bc90-2aeae9565700","Type":"ContainerDied","Data":"7017266d6aa959572e0676228a7ec3454eef4992716562e7586c611a875b105f"} Jul 24 13:46:23.638687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.638647 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7017266d6aa959572e0676228a7ec3454eef4992716562e7586c611a875b105f" Jul 24 13:46:23.640913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.640761 2558 generic.go:356] "Generic (PLEG): container finished" podID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerID="ee9de1aa24a84a0d13ac8a00564ed86cf618982bdfa5311d4c92f91fa5ed04b7" exitCode=0 Jul 24 13:46:23.640913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.640829 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" event={"ID":"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4","Type":"ContainerDied","Data":"ee9de1aa24a84a0d13ac8a00564ed86cf618982bdfa5311d4c92f91fa5ed04b7"} Jul 24 13:46:23.661479 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.661449 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:46:23.664535 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.664503 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:46:23.688706 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.688671 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2qj9\" (UniqueName: \"kubernetes.io/projected/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-kube-api-access-c2qj9\") pod \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " Jul 24 13:46:23.688909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.688731 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-plugins-config-volume\") pod \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " Jul 24 13:46:23.688909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.688767 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgljt\" (UniqueName: \"kubernetes.io/projected/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-kube-api-access-pgljt\") pod \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\" (UID: \"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4\") " Jul 24 13:46:23.688909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.688795 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-plugins-config-volume\") pod \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\" (UID: \"7f8a1dc2-2b62-4e09-bc90-2aeae9565700\") " Jul 24 13:46:23.689210 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.689185 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" (UID: "7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:46:23.689267 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.689240 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f8a1dc2-2b62-4e09-bc90-2aeae9565700" (UID: "7f8a1dc2-2b62-4e09-bc90-2aeae9565700"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:46:23.691015 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.690982 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-kube-api-access-pgljt" (OuterVolumeSpecName: "kube-api-access-pgljt") pod "7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" (UID: "7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4"). InnerVolumeSpecName "kube-api-access-pgljt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:46:23.691098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.691033 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-kube-api-access-c2qj9" (OuterVolumeSpecName: "kube-api-access-c2qj9") pod "7f8a1dc2-2b62-4e09-bc90-2aeae9565700" (UID: "7f8a1dc2-2b62-4e09-bc90-2aeae9565700"). InnerVolumeSpecName "kube-api-access-c2qj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:46:23.790006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.789963 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:23.790006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.789995 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgljt\" (UniqueName: \"kubernetes.io/projected/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4-kube-api-access-pgljt\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:23.790006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.790005 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:23.790006 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:23.790014 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2qj9\" (UniqueName: \"kubernetes.io/projected/7f8a1dc2-2b62-4e09-bc90-2aeae9565700-kube-api-access-c2qj9\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:24.647830 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.647737 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" Jul 24 13:46:24.647830 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.647781 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t" event={"ID":"7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4","Type":"ContainerDied","Data":"72e0b8935033662662a01f89f79588c69addf491cc82c700f5f57e5a0c2e3bfc"} Jul 24 13:46:24.648330 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.647851 2558 scope.go:117] "RemoveContainer" containerID="ee9de1aa24a84a0d13ac8a00564ed86cf618982bdfa5311d4c92f91fa5ed04b7" Jul 24 13:46:24.648330 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.647955 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n" Jul 24 13:46:24.677118 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.677070 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t"] Jul 24 13:46:24.682148 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.682113 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-50b298-7ccd5b4856-6sq6t"] Jul 24 13:46:24.693894 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.693851 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n"] Jul 24 13:46:24.697976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:24.697939 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-50b298-5967d94d5d-h9x4n"] Jul 24 13:46:25.196888 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:25.196851 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" path="/var/lib/kubelet/pods/7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4/volumes" Jul 24 13:46:25.197229 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:25.197215 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f8a1dc2-2b62-4e09-bc90-2aeae9565700" path="/var/lib/kubelet/pods/7f8a1dc2-2b62-4e09-bc90-2aeae9565700/volumes" Jul 24 13:46:27.927776 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:27.927743 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:27.938933 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:27.938896 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7"] Jul 24 13:46:27.942676 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:27.942643 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-50b298-2fvz7"] Jul 24 13:46:28.348162 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:28.348132 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:29.196885 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:29.196848 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755af00a-d780-437d-a9b9-c1e97468bd01" path="/var/lib/kubelet/pods/755af00a-d780-437d-a9b9-c1e97468bd01/volumes" Jul 24 13:46:33.315319 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:33.315271 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:46:34.126402 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:34.126346 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:34.193330 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:34.193293 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:34.405801 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:34.405697 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:34.645724 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:34.645691 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:45.263822 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.263784 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:45.322169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.322123 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z"] Jul 24 13:46:45.322440 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.322412 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" podUID="e9771fde-3567-47e4-8fce-59da1de5e188" containerName="payload-processing" containerID="cri-o://24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed" gracePeriod=30 Jul 24 13:46:45.383311 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.383248 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:45.525525 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.525440 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:45.594255 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.594221 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:46:45.694366 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.694321 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9771fde-3567-47e4-8fce-59da1de5e188-plugins-config-volume\") pod \"e9771fde-3567-47e4-8fce-59da1de5e188\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " Jul 24 13:46:45.694603 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.694548 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-794bp\" (UniqueName: \"kubernetes.io/projected/e9771fde-3567-47e4-8fce-59da1de5e188-kube-api-access-794bp\") pod \"e9771fde-3567-47e4-8fce-59da1de5e188\" (UID: \"e9771fde-3567-47e4-8fce-59da1de5e188\") " Jul 24 13:46:45.694778 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.694750 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9771fde-3567-47e4-8fce-59da1de5e188-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9771fde-3567-47e4-8fce-59da1de5e188" (UID: "e9771fde-3567-47e4-8fce-59da1de5e188"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:46:45.696625 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.696596 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9771fde-3567-47e4-8fce-59da1de5e188-kube-api-access-794bp" (OuterVolumeSpecName: "kube-api-access-794bp") pod "e9771fde-3567-47e4-8fce-59da1de5e188" (UID: "e9771fde-3567-47e4-8fce-59da1de5e188"). InnerVolumeSpecName "kube-api-access-794bp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:46:45.766639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.766606 2558 generic.go:356] "Generic (PLEG): container finished" podID="e9771fde-3567-47e4-8fce-59da1de5e188" containerID="24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed" exitCode=0 Jul 24 13:46:45.766845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.766661 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" Jul 24 13:46:45.766845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.766693 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" event={"ID":"e9771fde-3567-47e4-8fce-59da1de5e188","Type":"ContainerDied","Data":"24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed"} Jul 24 13:46:45.766845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.766757 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z" event={"ID":"e9771fde-3567-47e4-8fce-59da1de5e188","Type":"ContainerDied","Data":"9c76c7b79051cefc04a872902b0e21fb611a6f9a75d32e5ef23d566ae0dc67f9"} Jul 24 13:46:45.766845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.766787 2558 scope.go:117] "RemoveContainer" containerID="24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed" Jul 24 13:46:45.776145 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.776122 2558 scope.go:117] "RemoveContainer" containerID="24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed" Jul 24 13:46:45.776487 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:46:45.776461 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed\": container with ID starting with 24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed not found: ID does not exist" containerID="24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed" Jul 24 13:46:45.776594 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.776496 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed"} err="failed to get container status \"24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed\": rpc error: code = NotFound desc = could not find container \"24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed\": container with ID starting with 24caac51a0bf556669b0ab09790bed2912c26d64c7b106641705a7d18aa8e0ed not found: ID does not exist" Jul 24 13:46:45.795652 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.795601 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-794bp\" (UniqueName: \"kubernetes.io/projected/e9771fde-3567-47e4-8fce-59da1de5e188-kube-api-access-794bp\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:45.795652 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.795649 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9771fde-3567-47e4-8fce-59da1de5e188-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:46:45.797941 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.797905 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z"] Jul 24 13:46:45.806721 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:45.806684 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7918ee-7988b68c66-kg45z"] Jul 24 13:46:47.198505 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:47.198463 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9771fde-3567-47e4-8fce-59da1de5e188" path="/var/lib/kubelet/pods/e9771fde-3567-47e4-8fce-59da1de5e188/volumes" Jul 24 13:46:48.316215 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:48.316170 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:46:49.836231 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:49.836170 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:50.248750 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:50.248713 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:55.980287 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:55.980250 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:56.040193 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:56.040153 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:56.267345 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:56.267250 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:56.496439 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:56.496377 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:56.857564 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:56.857525 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:57.189055 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:57.188926 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:57.622576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:46:57.622534 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:00.038209 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:00.038142 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:00.131945 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:00.131902 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:00.209080 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:00.209048 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:00.638037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:00.638002 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:00.935428 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:00.935307 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:01.036696 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.036661 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:01.075951 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.075915 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84"] Jul 24 13:47:01.076570 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076546 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755af00a-d780-437d-a9b9-c1e97468bd01" containerName="revoke-keys" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076572 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="755af00a-d780-437d-a9b9-c1e97468bd01" containerName="revoke-keys" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076591 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076601 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076613 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9771fde-3567-47e4-8fce-59da1de5e188" containerName="payload-processing" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076622 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9771fde-3567-47e4-8fce-59da1de5e188" containerName="payload-processing" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076658 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f8a1dc2-2b62-4e09-bc90-2aeae9565700" containerName="payload-processing" Jul 24 13:47:01.076669 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076667 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f8a1dc2-2b62-4e09-bc90-2aeae9565700" containerName="payload-processing" Jul 24 13:47:01.076994 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076769 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f8a1dc2-2b62-4e09-bc90-2aeae9565700" containerName="payload-processing" Jul 24 13:47:01.076994 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076784 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9771fde-3567-47e4-8fce-59da1de5e188" containerName="payload-processing" Jul 24 13:47:01.076994 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076799 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="755af00a-d780-437d-a9b9-c1e97468bd01" containerName="revoke-keys" Jul 24 13:47:01.076994 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.076810 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a0833a6-5b1f-4bf2-9e7b-29f49c6f77a4" containerName="payload-pre-processing" Jul 24 13:47:01.080030 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.080000 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.082585 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.082554 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-502742\"" Jul 24 13:47:01.082736 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.082557 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-502742-dockercfg-2d6s2\"" Jul 24 13:47:01.089576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.089542 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84"] Jul 24 13:47:01.236707 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.236618 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwshm\" (UniqueName: \"kubernetes.io/projected/26edd1bc-7c42-42b4-8239-05f0f236e49b-kube-api-access-jwshm\") pod \"payload-processing-e2e-api-a-502742-c76b96b44-tpt84\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.236865 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.236731 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26edd1bc-7c42-42b4-8239-05f0f236e49b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-502742-c76b96b44-tpt84\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.338217 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.338166 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwshm\" (UniqueName: \"kubernetes.io/projected/26edd1bc-7c42-42b4-8239-05f0f236e49b-kube-api-access-jwshm\") pod \"payload-processing-e2e-api-a-502742-c76b96b44-tpt84\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.338472 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.338261 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26edd1bc-7c42-42b4-8239-05f0f236e49b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-502742-c76b96b44-tpt84\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.339020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.338997 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26edd1bc-7c42-42b4-8239-05f0f236e49b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-502742-c76b96b44-tpt84\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.352995 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.352964 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwshm\" (UniqueName: \"kubernetes.io/projected/26edd1bc-7c42-42b4-8239-05f0f236e49b-kube-api-access-jwshm\") pod \"payload-processing-e2e-api-a-502742-c76b96b44-tpt84\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.392025 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.391976 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.546102 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.546074 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84"] Jul 24 13:47:01.548659 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:47:01.548626 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26edd1bc_7c42_42b4_8239_05f0f236e49b.slice/crio-e016b96d26da4b8cfe13f21526e7f6a3e4b1ee9b19934b1020a0b5c2c7e61757 WatchSource:0}: Error finding container e016b96d26da4b8cfe13f21526e7f6a3e4b1ee9b19934b1020a0b5c2c7e61757: Status 404 returned error can't find the container with id e016b96d26da4b8cfe13f21526e7f6a3e4b1ee9b19934b1020a0b5c2c7e61757 Jul 24 13:47:01.854411 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.854331 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" event={"ID":"26edd1bc-7c42-42b4-8239-05f0f236e49b","Type":"ContainerStarted","Data":"e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34"} Jul 24 13:47:01.854598 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.854432 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" event={"ID":"26edd1bc-7c42-42b4-8239-05f0f236e49b","Type":"ContainerStarted","Data":"e016b96d26da4b8cfe13f21526e7f6a3e4b1ee9b19934b1020a0b5c2c7e61757"} Jul 24 13:47:01.854598 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.854461 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:01.873269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:01.873212 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" podStartSLOduration=0.873193627 podStartE2EDuration="873.193627ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:01.871113156 +0000 UTC m=+2285.235037247" watchObservedRunningTime="2026-07-24 13:47:01.873193627 +0000 UTC m=+2285.237117716" Jul 24 13:47:04.577133 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:04.577095 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:05.062645 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:05.062610 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:05.404136 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:05.404069 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:05.837631 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:05.837533 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:07.309782 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:07.309748 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:47:08.087278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.087245 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:08.191529 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.191490 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:08.295732 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.295687 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:08.395407 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.395349 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:08.797615 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.797520 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:08.899047 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.899013 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:08.906853 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.906810 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp"] Jul 24 13:47:08.910639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.910613 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:08.913181 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.913153 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mmpr\" (UniqueName: \"kubernetes.io/projected/9449011e-bfe5-4e07-b9d9-c9368838112a-kube-api-access-6mmpr\") pod \"payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:08.913272 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.913198 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9449011e-bfe5-4e07-b9d9-c9368838112a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:08.916292 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.916255 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-799cec-dockercfg-49d9p\"" Jul 24 13:47:08.916575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.916553 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-799cec\"" Jul 24 13:47:08.927583 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.927548 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp"] Jul 24 13:47:08.931984 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.931949 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p"] Jul 24 13:47:08.935477 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.935452 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:08.945258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:08.945106 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p"] Jul 24 13:47:09.013894 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.013850 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.014116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.013907 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mmpr\" (UniqueName: \"kubernetes.io/projected/9449011e-bfe5-4e07-b9d9-c9368838112a-kube-api-access-6mmpr\") pod \"payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:09.014116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.013934 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9449011e-bfe5-4e07-b9d9-c9368838112a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:09.014116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.013966 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvjcs\" (UniqueName: \"kubernetes.io/projected/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-kube-api-access-dvjcs\") pod \"payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.014763 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.014736 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9449011e-bfe5-4e07-b9d9-c9368838112a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:09.023463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.023430 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mmpr\" (UniqueName: \"kubernetes.io/projected/9449011e-bfe5-4e07-b9d9-c9368838112a-kube-api-access-6mmpr\") pod \"payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:09.115271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.115229 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvjcs\" (UniqueName: \"kubernetes.io/projected/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-kube-api-access-dvjcs\") pod \"payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.115505 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.115305 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.116094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.116073 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.124984 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.124955 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvjcs\" (UniqueName: \"kubernetes.io/projected/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-kube-api-access-dvjcs\") pod \"payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.221414 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.221346 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:09.247814 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.247768 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.390142 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.390046 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp"] Jul 24 13:47:09.396059 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:47:09.396027 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9449011e_bfe5_4e07_b9d9_c9368838112a.slice/crio-a84952fc76b45fb90dab0ceeb63c3deb88c7e208a62ea21fa8cfae3ed732c4ae WatchSource:0}: Error finding container a84952fc76b45fb90dab0ceeb63c3deb88c7e208a62ea21fa8cfae3ed732c4ae: Status 404 returned error can't find the container with id a84952fc76b45fb90dab0ceeb63c3deb88c7e208a62ea21fa8cfae3ed732c4ae Jul 24 13:47:09.421128 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.421100 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p"] Jul 24 13:47:09.432643 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:47:09.432605 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81513db_b3e2_4fbc_994e_fc740ce6c9c1.slice/crio-95c2f0adb35882a45aed1564e31e0a515c8cab328e2f8ea1ffdc08a2c640728b WatchSource:0}: Error finding container 95c2f0adb35882a45aed1564e31e0a515c8cab328e2f8ea1ffdc08a2c640728b: Status 404 returned error can't find the container with id 95c2f0adb35882a45aed1564e31e0a515c8cab328e2f8ea1ffdc08a2c640728b Jul 24 13:47:09.898860 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.898817 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" event={"ID":"9449011e-bfe5-4e07-b9d9-c9368838112a","Type":"ContainerStarted","Data":"71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba"} Jul 24 13:47:09.899065 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.898980 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:09.899065 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.899019 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" event={"ID":"9449011e-bfe5-4e07-b9d9-c9368838112a","Type":"ContainerStarted","Data":"a84952fc76b45fb90dab0ceeb63c3deb88c7e208a62ea21fa8cfae3ed732c4ae"} Jul 24 13:47:09.900573 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.900545 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" event={"ID":"c81513db-b3e2-4fbc-994e-fc740ce6c9c1","Type":"ContainerStarted","Data":"7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37"} Jul 24 13:47:09.900704 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.900595 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" event={"ID":"c81513db-b3e2-4fbc-994e-fc740ce6c9c1","Type":"ContainerStarted","Data":"95c2f0adb35882a45aed1564e31e0a515c8cab328e2f8ea1ffdc08a2c640728b"} Jul 24 13:47:09.900704 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.900630 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:09.920686 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.920620 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" podStartSLOduration=1.9206003200000001 podStartE2EDuration="1.92060032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:09.917876966 +0000 UTC m=+2293.281801056" watchObservedRunningTime="2026-07-24 13:47:09.92060032 +0000 UTC m=+2293.284524409" Jul 24 13:47:09.936287 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:09.936220 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" podStartSLOduration=1.936199957 podStartE2EDuration="1.936199957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:09.933666923 +0000 UTC m=+2293.297591013" watchObservedRunningTime="2026-07-24 13:47:09.936199957 +0000 UTC m=+2293.300124047" Jul 24 13:47:12.610859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:12.610817 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:12.862054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:12.861954 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:20.909143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:20.909110 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:47:20.909614 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:20.909181 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:47:23.421525 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:23.421484 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:47:27.470560 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:27.470518 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:36.710014 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:36.709975 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:36.742842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:36.742795 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84"] Jul 24 13:47:36.743273 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:36.743067 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" podUID="26edd1bc-7c42-42b4-8239-05f0f236e49b" containerName="payload-processing" containerID="cri-o://e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34" gracePeriod=30 Jul 24 13:47:36.810041 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:36.810002 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:36.909289 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:36.909252 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:37.021731 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.021701 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:37.051206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.051170 2558 generic.go:356] "Generic (PLEG): container finished" podID="26edd1bc-7c42-42b4-8239-05f0f236e49b" containerID="e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34" exitCode=0 Jul 24 13:47:37.051206 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.051214 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" Jul 24 13:47:37.051463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.051239 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" event={"ID":"26edd1bc-7c42-42b4-8239-05f0f236e49b","Type":"ContainerDied","Data":"e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34"} Jul 24 13:47:37.051463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.051289 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84" event={"ID":"26edd1bc-7c42-42b4-8239-05f0f236e49b","Type":"ContainerDied","Data":"e016b96d26da4b8cfe13f21526e7f6a3e4b1ee9b19934b1020a0b5c2c7e61757"} Jul 24 13:47:37.051463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.051316 2558 scope.go:117] "RemoveContainer" containerID="e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34" Jul 24 13:47:37.062202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.062169 2558 scope.go:117] "RemoveContainer" containerID="e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34" Jul 24 13:47:37.062654 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:47:37.062626 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34\": container with ID starting with e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34 not found: ID does not exist" containerID="e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34" Jul 24 13:47:37.062838 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.062668 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34"} err="failed to get container status \"e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34\": rpc error: code = NotFound desc = could not find container \"e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34\": container with ID starting with e1ff42329bdc2bae7363e240b6eb75cc1b1ee89d32f563dce88ef54a6bcf8a34 not found: ID does not exist" Jul 24 13:47:37.170592 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.170560 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26edd1bc-7c42-42b4-8239-05f0f236e49b-plugins-config-volume\") pod \"26edd1bc-7c42-42b4-8239-05f0f236e49b\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " Jul 24 13:47:37.170780 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.170617 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwshm\" (UniqueName: \"kubernetes.io/projected/26edd1bc-7c42-42b4-8239-05f0f236e49b-kube-api-access-jwshm\") pod \"26edd1bc-7c42-42b4-8239-05f0f236e49b\" (UID: \"26edd1bc-7c42-42b4-8239-05f0f236e49b\") " Jul 24 13:47:37.171137 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.171104 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26edd1bc-7c42-42b4-8239-05f0f236e49b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26edd1bc-7c42-42b4-8239-05f0f236e49b" (UID: "26edd1bc-7c42-42b4-8239-05f0f236e49b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:47:37.173035 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.172999 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26edd1bc-7c42-42b4-8239-05f0f236e49b-kube-api-access-jwshm" (OuterVolumeSpecName: "kube-api-access-jwshm") pod "26edd1bc-7c42-42b4-8239-05f0f236e49b" (UID: "26edd1bc-7c42-42b4-8239-05f0f236e49b"). InnerVolumeSpecName "kube-api-access-jwshm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:47:37.272153 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.272062 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwshm\" (UniqueName: \"kubernetes.io/projected/26edd1bc-7c42-42b4-8239-05f0f236e49b-kube-api-access-jwshm\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:47:37.272153 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.272094 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26edd1bc-7c42-42b4-8239-05f0f236e49b-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:47:37.368534 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.368495 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84"] Jul 24 13:47:37.374114 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:37.374078 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-502742-c76b96b44-tpt84"] Jul 24 13:47:39.197503 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:39.197453 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26edd1bc-7c42-42b4-8239-05f0f236e49b" path="/var/lib/kubelet/pods/26edd1bc-7c42-42b4-8239-05f0f236e49b/volumes" Jul 24 13:47:39.731030 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:39.730992 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:47:40.935556 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:40.935512 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:41.356864 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:41.356830 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:47.152174 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:47.152130 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:47.211691 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:47.211651 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:47.439405 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:47.439292 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:47.684939 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:47.684904 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:47:57.357752 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:47:57.357650 2558 scope.go:117] "RemoveContainer" containerID="f3a4e525a9123a472a1a870de3ef021df17fdccd88280795037c82cd402aedb7" Jul 24 13:48:03.264956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.264914 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:03.288821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.288785 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p"] Jul 24 13:48:03.289083 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.289057 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" podUID="c81513db-b3e2-4fbc-994e-fc740ce6c9c1" containerName="payload-processing" containerID="cri-o://7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37" gracePeriod=30 Jul 24 13:48:03.296211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.296172 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp"] Jul 24 13:48:03.296466 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.296428 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" podUID="9449011e-bfe5-4e07-b9d9-c9368838112a" containerName="payload-pre-processing" containerID="cri-o://71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba" gracePeriod=30 Jul 24 13:48:03.327459 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.327427 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:03.386434 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.386379 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:03.584276 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.584248 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:48:03.587067 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.587040 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:48:03.604954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.604916 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-plugins-config-volume\") pod \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " Jul 24 13:48:03.605172 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.604966 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mmpr\" (UniqueName: \"kubernetes.io/projected/9449011e-bfe5-4e07-b9d9-c9368838112a-kube-api-access-6mmpr\") pod \"9449011e-bfe5-4e07-b9d9-c9368838112a\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " Jul 24 13:48:03.605172 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.605048 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9449011e-bfe5-4e07-b9d9-c9368838112a-plugins-config-volume\") pod \"9449011e-bfe5-4e07-b9d9-c9368838112a\" (UID: \"9449011e-bfe5-4e07-b9d9-c9368838112a\") " Jul 24 13:48:03.605172 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.605116 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvjcs\" (UniqueName: \"kubernetes.io/projected/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-kube-api-access-dvjcs\") pod \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\" (UID: \"c81513db-b3e2-4fbc-994e-fc740ce6c9c1\") " Jul 24 13:48:03.605454 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.605365 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c81513db-b3e2-4fbc-994e-fc740ce6c9c1" (UID: "c81513db-b3e2-4fbc-994e-fc740ce6c9c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:48:03.605810 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.605769 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9449011e-bfe5-4e07-b9d9-c9368838112a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9449011e-bfe5-4e07-b9d9-c9368838112a" (UID: "9449011e-bfe5-4e07-b9d9-c9368838112a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:48:03.607813 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.607778 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9449011e-bfe5-4e07-b9d9-c9368838112a-kube-api-access-6mmpr" (OuterVolumeSpecName: "kube-api-access-6mmpr") pod "9449011e-bfe5-4e07-b9d9-c9368838112a" (UID: "9449011e-bfe5-4e07-b9d9-c9368838112a"). InnerVolumeSpecName "kube-api-access-6mmpr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:48:03.607928 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.607791 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-kube-api-access-dvjcs" (OuterVolumeSpecName: "kube-api-access-dvjcs") pod "c81513db-b3e2-4fbc-994e-fc740ce6c9c1" (UID: "c81513db-b3e2-4fbc-994e-fc740ce6c9c1"). InnerVolumeSpecName "kube-api-access-dvjcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:48:03.706710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.706664 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9449011e-bfe5-4e07-b9d9-c9368838112a-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:48:03.706710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.706700 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvjcs\" (UniqueName: \"kubernetes.io/projected/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-kube-api-access-dvjcs\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:48:03.706710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.706710 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c81513db-b3e2-4fbc-994e-fc740ce6c9c1-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:48:03.706710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:03.706718 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mmpr\" (UniqueName: \"kubernetes.io/projected/9449011e-bfe5-4e07-b9d9-c9368838112a-kube-api-access-6mmpr\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:48:04.200618 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.200577 2558 generic.go:356] "Generic (PLEG): container finished" podID="c81513db-b3e2-4fbc-994e-fc740ce6c9c1" containerID="7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37" exitCode=0 Jul 24 13:48:04.200810 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.200633 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" Jul 24 13:48:04.200810 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.200660 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" event={"ID":"c81513db-b3e2-4fbc-994e-fc740ce6c9c1","Type":"ContainerDied","Data":"7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37"} Jul 24 13:48:04.200810 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.200714 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p" event={"ID":"c81513db-b3e2-4fbc-994e-fc740ce6c9c1","Type":"ContainerDied","Data":"95c2f0adb35882a45aed1564e31e0a515c8cab328e2f8ea1ffdc08a2c640728b"} Jul 24 13:48:04.200810 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.200745 2558 scope.go:117] "RemoveContainer" containerID="7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37" Jul 24 13:48:04.202148 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.202123 2558 generic.go:356] "Generic (PLEG): container finished" podID="9449011e-bfe5-4e07-b9d9-c9368838112a" containerID="71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba" exitCode=0 Jul 24 13:48:04.202237 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.202172 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" event={"ID":"9449011e-bfe5-4e07-b9d9-c9368838112a","Type":"ContainerDied","Data":"71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba"} Jul 24 13:48:04.202237 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.202211 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" event={"ID":"9449011e-bfe5-4e07-b9d9-c9368838112a","Type":"ContainerDied","Data":"a84952fc76b45fb90dab0ceeb63c3deb88c7e208a62ea21fa8cfae3ed732c4ae"} Jul 24 13:48:04.202344 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.202255 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp" Jul 24 13:48:04.209934 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.209909 2558 scope.go:117] "RemoveContainer" containerID="7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37" Jul 24 13:48:04.210266 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:48:04.210243 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37\": container with ID starting with 7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37 not found: ID does not exist" containerID="7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37" Jul 24 13:48:04.210340 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.210279 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37"} err="failed to get container status \"7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37\": rpc error: code = NotFound desc = could not find container \"7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37\": container with ID starting with 7fc5418f3eef70c8a84a794d86f19f6f7d4e5d396559b9dec4a2669807c22b37 not found: ID does not exist" Jul 24 13:48:04.210340 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.210296 2558 scope.go:117] "RemoveContainer" containerID="71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba" Jul 24 13:48:04.219187 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.219169 2558 scope.go:117] "RemoveContainer" containerID="71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba" Jul 24 13:48:04.219519 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:48:04.219497 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba\": container with ID starting with 71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba not found: ID does not exist" containerID="71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba" Jul 24 13:48:04.219594 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.219526 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba"} err="failed to get container status \"71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba\": rpc error: code = NotFound desc = could not find container \"71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba\": container with ID starting with 71976f6e82d3e9aee96acfb22a5f71bc737cbfabce1d256785d7f94cb00e90ba not found: ID does not exist" Jul 24 13:48:04.227791 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.227760 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp"] Jul 24 13:48:04.230339 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.230310 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-799cec-b4d79bf6d-v4dcp"] Jul 24 13:48:04.240178 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.240146 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p"] Jul 24 13:48:04.246116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:04.246088 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-799cec-5b7f8b9fbb-qcz5p"] Jul 24 13:48:05.196529 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:05.196482 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9449011e-bfe5-4e07-b9d9-c9368838112a" path="/var/lib/kubelet/pods/9449011e-bfe5-4e07-b9d9-c9368838112a/volumes" Jul 24 13:48:05.197051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:05.197026 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c81513db-b3e2-4fbc-994e-fc740ce6c9c1" path="/var/lib/kubelet/pods/c81513db-b3e2-4fbc-994e-fc740ce6c9c1/volumes" Jul 24 13:48:07.893999 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:07.893966 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:08.301150 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:08.301018 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:08.316265 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:08.316221 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:48:14.120126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:14.120080 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:14.187327 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:14.187285 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:14.406038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:14.405942 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:14.658803 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:14.658689 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:15.018065 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:15.017972 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:15.360733 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:15.360701 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:15.548173 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:15.548142 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:18.443644 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:18.443600 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:18.542795 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:18.542757 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:18.693396 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:18.693350 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:18.877331 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:18.877294 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:19.160048 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.159943 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:19.262529 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.262489 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:19.263690 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.263662 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt"] Jul 24 13:48:19.264104 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264086 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9449011e-bfe5-4e07-b9d9-c9368838112a" containerName="payload-pre-processing" Jul 24 13:48:19.264183 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264106 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9449011e-bfe5-4e07-b9d9-c9368838112a" containerName="payload-pre-processing" Jul 24 13:48:19.264183 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264142 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81513db-b3e2-4fbc-994e-fc740ce6c9c1" containerName="payload-processing" Jul 24 13:48:19.264183 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264151 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81513db-b3e2-4fbc-994e-fc740ce6c9c1" containerName="payload-processing" Jul 24 13:48:19.264183 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264165 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26edd1bc-7c42-42b4-8239-05f0f236e49b" containerName="payload-processing" Jul 24 13:48:19.264183 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264174 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="26edd1bc-7c42-42b4-8239-05f0f236e49b" containerName="payload-processing" Jul 24 13:48:19.264445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264281 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81513db-b3e2-4fbc-994e-fc740ce6c9c1" containerName="payload-processing" Jul 24 13:48:19.264445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264296 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9449011e-bfe5-4e07-b9d9-c9368838112a" containerName="payload-pre-processing" Jul 24 13:48:19.264445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.264307 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="26edd1bc-7c42-42b4-8239-05f0f236e49b" containerName="payload-processing" Jul 24 13:48:19.268951 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.268925 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.271750 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.271726 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-934481-dockercfg-7lv4t\"" Jul 24 13:48:19.271879 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.271860 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-934481\"" Jul 24 13:48:19.277823 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.277797 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt"] Jul 24 13:48:19.302643 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.302604 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg"] Jul 24 13:48:19.306191 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.306168 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.318979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.318945 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg"] Jul 24 13:48:19.347037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.346991 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bplbt\" (UniqueName: \"kubernetes.io/projected/edc50b40-f120-41ec-a398-9e8b74ee7f1d-kube-api-access-bplbt\") pod \"payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.347218 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.347086 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.347218 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.347115 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqt9\" (UniqueName: \"kubernetes.io/projected/85917efa-01af-4714-9f43-1da7aa37adb7-kube-api-access-dmqt9\") pod \"payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.347218 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.347207 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc50b40-f120-41ec-a398-9e8b74ee7f1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.447869 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.447757 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bplbt\" (UniqueName: \"kubernetes.io/projected/edc50b40-f120-41ec-a398-9e8b74ee7f1d-kube-api-access-bplbt\") pod \"payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.447869 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.447821 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.447869 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.447844 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqt9\" (UniqueName: \"kubernetes.io/projected/85917efa-01af-4714-9f43-1da7aa37adb7-kube-api-access-dmqt9\") pod \"payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.448537 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.447906 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc50b40-f120-41ec-a398-9e8b74ee7f1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.448678 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.448653 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.448828 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.448796 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc50b40-f120-41ec-a398-9e8b74ee7f1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.455663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.455624 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bplbt\" (UniqueName: \"kubernetes.io/projected/edc50b40-f120-41ec-a398-9e8b74ee7f1d-kube-api-access-bplbt\") pod \"payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.455814 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.455697 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqt9\" (UniqueName: \"kubernetes.io/projected/85917efa-01af-4714-9f43-1da7aa37adb7-kube-api-access-dmqt9\") pod \"payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.580760 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.580716 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:19.616248 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.616214 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:19.769576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.769533 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt"] Jul 24 13:48:19.773658 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:48:19.773604 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc50b40_f120_41ec_a398_9e8b74ee7f1d.slice/crio-da8bf97023d0302b1d5978022a2d07d62efa99d72b2683f1b8ca32c0409f9602 WatchSource:0}: Error finding container da8bf97023d0302b1d5978022a2d07d62efa99d72b2683f1b8ca32c0409f9602: Status 404 returned error can't find the container with id da8bf97023d0302b1d5978022a2d07d62efa99d72b2683f1b8ca32c0409f9602 Jul 24 13:48:19.804066 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:19.804024 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg"] Jul 24 13:48:19.807192 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:48:19.807157 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85917efa_01af_4714_9f43_1da7aa37adb7.slice/crio-6ab263e1001ee6ce6e241f004c0155ad6a093c6fe8d675e2ba5b7081191d0dc7 WatchSource:0}: Error finding container 6ab263e1001ee6ce6e241f004c0155ad6a093c6fe8d675e2ba5b7081191d0dc7: Status 404 returned error can't find the container with id 6ab263e1001ee6ce6e241f004c0155ad6a093c6fe8d675e2ba5b7081191d0dc7 Jul 24 13:48:20.291884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.291845 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" event={"ID":"85917efa-01af-4714-9f43-1da7aa37adb7","Type":"ContainerStarted","Data":"dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1"} Jul 24 13:48:20.292108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.291914 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" event={"ID":"85917efa-01af-4714-9f43-1da7aa37adb7","Type":"ContainerStarted","Data":"6ab263e1001ee6ce6e241f004c0155ad6a093c6fe8d675e2ba5b7081191d0dc7"} Jul 24 13:48:20.292108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.291951 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:20.293615 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.293588 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" event={"ID":"edc50b40-f120-41ec-a398-9e8b74ee7f1d","Type":"ContainerStarted","Data":"dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39"} Jul 24 13:48:20.293756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.293633 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" event={"ID":"edc50b40-f120-41ec-a398-9e8b74ee7f1d","Type":"ContainerStarted","Data":"da8bf97023d0302b1d5978022a2d07d62efa99d72b2683f1b8ca32c0409f9602"} Jul 24 13:48:20.293756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.293707 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:20.309602 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.309543 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" podStartSLOduration=1.309527396 podStartE2EDuration="1.309527396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:20.308193842 +0000 UTC m=+2363.672117931" watchObservedRunningTime="2026-07-24 13:48:20.309527396 +0000 UTC m=+2363.673451486" Jul 24 13:48:20.326914 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:20.326852 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" podStartSLOduration=1.326833899 podStartE2EDuration="1.326833899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:20.324821253 +0000 UTC m=+2363.688745343" watchObservedRunningTime="2026-07-24 13:48:20.326833899 +0000 UTC m=+2363.690757988" Jul 24 13:48:22.978661 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:22.978616 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:23.333595 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:23.333495 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:23.690048 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:23.689978 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:23.917185 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:23.917146 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:24.351782 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:24.351736 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:48:26.541708 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:26.541656 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:26.651736 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:26.651694 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:26.741576 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:26.741541 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:27.169213 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.169179 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:27.358430 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.358378 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:27.455275 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.455180 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:27.541400 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.541351 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6"] Jul 24 13:48:27.546211 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.546183 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.549095 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.549063 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-0a0df7\"" Jul 24 13:48:27.549249 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.549113 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-0a0df7-dockercfg-scmx2\"" Jul 24 13:48:27.568346 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.568299 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6"] Jul 24 13:48:27.624008 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.623967 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86tsp\" (UniqueName: \"kubernetes.io/projected/65619dfc-6f77-4b5f-82a4-8f056d672f38-kube-api-access-86tsp\") pod \"payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.624176 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.624033 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65619dfc-6f77-4b5f-82a4-8f056d672f38-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.725038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.724939 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86tsp\" (UniqueName: \"kubernetes.io/projected/65619dfc-6f77-4b5f-82a4-8f056d672f38-kube-api-access-86tsp\") pod \"payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.725038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.724999 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65619dfc-6f77-4b5f-82a4-8f056d672f38-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.725796 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.725774 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65619dfc-6f77-4b5f-82a4-8f056d672f38-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.735620 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.735590 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86tsp\" (UniqueName: \"kubernetes.io/projected/65619dfc-6f77-4b5f-82a4-8f056d672f38-kube-api-access-86tsp\") pod \"payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:27.870035 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:27.869987 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:28.023223 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:28.023143 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6"] Jul 24 13:48:28.025729 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:48:28.025692 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65619dfc_6f77_4b5f_82a4_8f056d672f38.slice/crio-78d3ca812f502263a8a26fcdadf8f041812b781e2db2440f20d53f318ae02e57 WatchSource:0}: Error finding container 78d3ca812f502263a8a26fcdadf8f041812b781e2db2440f20d53f318ae02e57: Status 404 returned error can't find the container with id 78d3ca812f502263a8a26fcdadf8f041812b781e2db2440f20d53f318ae02e57 Jul 24 13:48:28.340977 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:28.340882 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" event={"ID":"65619dfc-6f77-4b5f-82a4-8f056d672f38","Type":"ContainerStarted","Data":"033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9"} Jul 24 13:48:28.340977 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:28.340934 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" event={"ID":"65619dfc-6f77-4b5f-82a4-8f056d672f38","Type":"ContainerStarted","Data":"78d3ca812f502263a8a26fcdadf8f041812b781e2db2440f20d53f318ae02e57"} Jul 24 13:48:28.341165 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:28.340975 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:31.073248 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:31.073208 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:31.305089 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:31.305057 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:48:31.305275 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:31.305144 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:48:31.322893 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:31.322830 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" podStartSLOduration=4.322807775 podStartE2EDuration="4.322807775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:28.357426828 +0000 UTC m=+2371.721350922" watchObservedRunningTime="2026-07-24 13:48:31.322807775 +0000 UTC m=+2374.686731865" Jul 24 13:48:37.113235 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:37.113189 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:48:39.349231 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:39.349195 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:48:50.946048 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:50.946012 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:54.731640 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:54.731598 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:48:57.264109 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:57.264079 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:48:57.272701 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:48:57.272671 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:49:19.151056 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:49:19.151013 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:23.011735 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:49:23.011693 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:49:40.715757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:49:40.715716 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:49:57.112278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:49:57.112235 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:50:14.010467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:14.010424 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:50:18.906348 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:18.906310 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:18.957111 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:18.957070 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:30.320627 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:30.320580 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:50:48.610896 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:48.610856 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:50:57.178887 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:57.178801 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:57.409192 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:50:57.409159 2558 scope.go:117] "RemoveContainer" containerID="db929c4d7622590a510e181bcb87ff7ad3c1beaf50c87995c33e7d7045d9c63a" Jul 24 13:51:05.219133 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:05.219095 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:51:15.577168 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:15.577131 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:17.240693 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:17.240658 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:18.271853 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:18.271815 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:22.211981 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:22.211942 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:51:30.723917 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:30.723879 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:32.982277 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:32.982235 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:35.420842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:35.420800 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:51:47.222360 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:47.222321 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:51.113171 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:51:51.113128 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:52:05.913748 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:05.913701 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:52:22.414757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:22.414672 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:52:28.362757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:28.362721 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:29.316316 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:29.316280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:41.506846 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:41.506807 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:52:50.111166 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:50.111116 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:54.932723 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:54.932685 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:52:57.439155 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:52:57.439120 2558 scope.go:117] "RemoveContainer" containerID="487c59bca0e4172e389fb7b4d72b33aefdfdd592b76ba1a4ef35e811d4973edd" Jul 24 13:53:50.048071 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:53:50.047977 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:53.917245 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:53:53.917207 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:53:55.611560 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:53:55.611524 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:55.666511 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:53:55.666466 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:57.296443 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:53:57.296414 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:53:57.309988 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:53:57.309958 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:54:07.912470 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:07.912425 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:54:29.630116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:29.630067 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:54:34.011697 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:34.011657 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:34.966891 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:34.966855 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:36.131068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.131028 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7646-139825883266736"] Jul 24 13:54:36.134553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.134528 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:36.137022 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.136991 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-b7cxb\"" Jul 24 13:54:36.138008 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.137979 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883266736"] Jul 24 13:54:36.317169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.317112 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlm5w\" (UniqueName: \"kubernetes.io/projected/22889d5a-cace-4471-9bf2-7f26be395412-kube-api-access-jlm5w\") pod \"test-curl-7646-139825883266736\" (UID: \"22889d5a-cace-4471-9bf2-7f26be395412\") " pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:36.417811 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.417700 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlm5w\" (UniqueName: \"kubernetes.io/projected/22889d5a-cace-4471-9bf2-7f26be395412-kube-api-access-jlm5w\") pod \"test-curl-7646-139825883266736\" (UID: \"22889d5a-cace-4471-9bf2-7f26be395412\") " pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:36.426607 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.426574 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlm5w\" (UniqueName: \"kubernetes.io/projected/22889d5a-cace-4471-9bf2-7f26be395412-kube-api-access-jlm5w\") pod \"test-curl-7646-139825883266736\" (UID: \"22889d5a-cace-4471-9bf2-7f26be395412\") " pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:36.446534 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.446497 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:36.596622 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.596592 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883266736"] Jul 24 13:54:36.599531 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:54:36.599501 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22889d5a_cace_4471_9bf2_7f26be395412.slice/crio-e55d0dbfcb16577814dd870109a02df62838291dca15c7baf9df4168f8c93b4a WatchSource:0}: Error finding container e55d0dbfcb16577814dd870109a02df62838291dca15c7baf9df4168f8c93b4a: Status 404 returned error can't find the container with id e55d0dbfcb16577814dd870109a02df62838291dca15c7baf9df4168f8c93b4a Jul 24 13:54:36.600754 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:36.600730 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:54:37.329819 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:37.329776 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883266736" event={"ID":"22889d5a-cace-4471-9bf2-7f26be395412","Type":"ContainerStarted","Data":"e55d0dbfcb16577814dd870109a02df62838291dca15c7baf9df4168f8c93b4a"} Jul 24 13:54:38.336613 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:38.336581 2558 generic.go:356] "Generic (PLEG): container finished" podID="22889d5a-cace-4471-9bf2-7f26be395412" containerID="a008f16cc4139e8069beeabe7e67ed98c0925e191838014f18b8b194c18d2c4b" exitCode=0 Jul 24 13:54:38.337016 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:38.336629 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883266736" event={"ID":"22889d5a-cace-4471-9bf2-7f26be395412","Type":"ContainerDied","Data":"a008f16cc4139e8069beeabe7e67ed98c0925e191838014f18b8b194c18d2c4b"} Jul 24 13:54:39.489246 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.489214 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:39.509966 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.509939 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7646-139825883266736_22889d5a-cace-4471-9bf2-7f26be395412/test-curl-7646-139825883266736/0.log" Jul 24 13:54:39.534805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.534768 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7646-139825883266736"] Jul 24 13:54:39.538513 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.538473 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7646-139825883266736"] Jul 24 13:54:39.645702 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.645660 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlm5w\" (UniqueName: \"kubernetes.io/projected/22889d5a-cace-4471-9bf2-7f26be395412-kube-api-access-jlm5w\") pod \"22889d5a-cace-4471-9bf2-7f26be395412\" (UID: \"22889d5a-cace-4471-9bf2-7f26be395412\") " Jul 24 13:54:39.647951 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.647909 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22889d5a-cace-4471-9bf2-7f26be395412-kube-api-access-jlm5w" (OuterVolumeSpecName: "kube-api-access-jlm5w") pod "22889d5a-cace-4471-9bf2-7f26be395412" (UID: "22889d5a-cace-4471-9bf2-7f26be395412"). InnerVolumeSpecName "kube-api-access-jlm5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:39.686910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.686863 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7646-139825883264560"] Jul 24 13:54:39.687313 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.687297 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22889d5a-cace-4471-9bf2-7f26be395412" containerName="test-curl-7646-139825883266736" Jul 24 13:54:39.687361 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.687314 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="22889d5a-cace-4471-9bf2-7f26be395412" containerName="test-curl-7646-139825883266736" Jul 24 13:54:39.687431 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.687378 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="22889d5a-cace-4471-9bf2-7f26be395412" containerName="test-curl-7646-139825883266736" Jul 24 13:54:39.690523 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.690502 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:39.694860 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.694832 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883264560"] Jul 24 13:54:39.747068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.747026 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlm5w\" (UniqueName: \"kubernetes.io/projected/22889d5a-cace-4471-9bf2-7f26be395412-kube-api-access-jlm5w\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:54:39.848233 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.848189 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmpcz\" (UniqueName: \"kubernetes.io/projected/220c07fe-33b6-470b-9a0f-f3ddcd70377c-kube-api-access-vmpcz\") pod \"test-curl-7646-139825883264560\" (UID: \"220c07fe-33b6-470b-9a0f-f3ddcd70377c\") " pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:39.948819 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.948723 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmpcz\" (UniqueName: \"kubernetes.io/projected/220c07fe-33b6-470b-9a0f-f3ddcd70377c-kube-api-access-vmpcz\") pod \"test-curl-7646-139825883264560\" (UID: \"220c07fe-33b6-470b-9a0f-f3ddcd70377c\") " pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:39.957327 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:39.957292 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmpcz\" (UniqueName: \"kubernetes.io/projected/220c07fe-33b6-470b-9a0f-f3ddcd70377c-kube-api-access-vmpcz\") pod \"test-curl-7646-139825883264560\" (UID: \"220c07fe-33b6-470b-9a0f-f3ddcd70377c\") " pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:40.001518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:40.001465 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:40.144794 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:40.144718 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883264560"] Jul 24 13:54:40.146966 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:54:40.146934 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod220c07fe_33b6_470b_9a0f_f3ddcd70377c.slice/crio-f0856c3da14b1d4e393af21dbc0f01036a89458b7cf10705a9dcb285a6126652 WatchSource:0}: Error finding container f0856c3da14b1d4e393af21dbc0f01036a89458b7cf10705a9dcb285a6126652: Status 404 returned error can't find the container with id f0856c3da14b1d4e393af21dbc0f01036a89458b7cf10705a9dcb285a6126652 Jul 24 13:54:40.348955 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:40.348923 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883264560" event={"ID":"220c07fe-33b6-470b-9a0f-f3ddcd70377c","Type":"ContainerStarted","Data":"f0856c3da14b1d4e393af21dbc0f01036a89458b7cf10705a9dcb285a6126652"} Jul 24 13:54:40.350285 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:40.350263 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e55d0dbfcb16577814dd870109a02df62838291dca15c7baf9df4168f8c93b4a" Jul 24 13:54:40.350438 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:40.350279 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:40.362299 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:54:40.362251 2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7646-139825883266736\" is forbidden: User \"system:node:ip-10-0-139-210.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-210.ec2.internal' and this object" podUID="22889d5a-cace-4471-9bf2-7f26be395412" pod="opendatahub/test-curl-7646-139825883266736" Jul 24 13:54:41.196795 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:41.196757 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22889d5a-cace-4471-9bf2-7f26be395412" path="/var/lib/kubelet/pods/22889d5a-cace-4471-9bf2-7f26be395412/volumes" Jul 24 13:54:41.362295 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:41.362240 2558 generic.go:356] "Generic (PLEG): container finished" podID="220c07fe-33b6-470b-9a0f-f3ddcd70377c" containerID="64b886ac38abd2b356162e3cfbf1506f7113547dd3c51cea3d676d9829148d92" exitCode=0 Jul 24 13:54:41.362501 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:41.362316 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883264560" event={"ID":"220c07fe-33b6-470b-9a0f-f3ddcd70377c","Type":"ContainerDied","Data":"64b886ac38abd2b356162e3cfbf1506f7113547dd3c51cea3d676d9829148d92"} Jul 24 13:54:42.517156 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.517126 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:42.537730 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.537692 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7646-139825883264560_220c07fe-33b6-470b-9a0f-f3ddcd70377c/test-curl-7646-139825883264560/0.log" Jul 24 13:54:42.560061 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.560015 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7646-139825883264560"] Jul 24 13:54:42.563862 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.563826 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7646-139825883264560"] Jul 24 13:54:42.673143 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.673045 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmpcz\" (UniqueName: \"kubernetes.io/projected/220c07fe-33b6-470b-9a0f-f3ddcd70377c-kube-api-access-vmpcz\") pod \"220c07fe-33b6-470b-9a0f-f3ddcd70377c\" (UID: \"220c07fe-33b6-470b-9a0f-f3ddcd70377c\") " Jul 24 13:54:42.675106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.675075 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220c07fe-33b6-470b-9a0f-f3ddcd70377c-kube-api-access-vmpcz" (OuterVolumeSpecName: "kube-api-access-vmpcz") pod "220c07fe-33b6-470b-9a0f-f3ddcd70377c" (UID: "220c07fe-33b6-470b-9a0f-f3ddcd70377c"). InnerVolumeSpecName "kube-api-access-vmpcz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:42.747335 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.747297 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:42.747776 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.747761 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220c07fe-33b6-470b-9a0f-f3ddcd70377c" containerName="test-curl-7646-139825883264560" Jul 24 13:54:42.747820 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.747778 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="220c07fe-33b6-470b-9a0f-f3ddcd70377c" containerName="test-curl-7646-139825883264560" Jul 24 13:54:42.747858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.747847 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="220c07fe-33b6-470b-9a0f-f3ddcd70377c" containerName="test-curl-7646-139825883264560" Jul 24 13:54:42.750877 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.750856 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:42.756765 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.756737 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:42.774169 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.774127 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmpcz\" (UniqueName: \"kubernetes.io/projected/220c07fe-33b6-470b-9a0f-f3ddcd70377c-kube-api-access-vmpcz\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:54:42.875495 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.875451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf97m\" (UniqueName: \"kubernetes.io/projected/46916001-5a2c-4be7-9169-c523e8ec418f-kube-api-access-kf97m\") pod \"test-curl-7646-139825882224368\" (UID: \"46916001-5a2c-4be7-9169-c523e8ec418f\") " pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:42.976372 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.976275 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf97m\" (UniqueName: \"kubernetes.io/projected/46916001-5a2c-4be7-9169-c523e8ec418f-kube-api-access-kf97m\") pod \"test-curl-7646-139825882224368\" (UID: \"46916001-5a2c-4be7-9169-c523e8ec418f\") " pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:42.985429 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:42.985352 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf97m\" (UniqueName: \"kubernetes.io/projected/46916001-5a2c-4be7-9169-c523e8ec418f-kube-api-access-kf97m\") pod \"test-curl-7646-139825882224368\" (UID: \"46916001-5a2c-4be7-9169-c523e8ec418f\") " pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:43.061166 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:43.061122 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:43.197845 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:43.197807 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220c07fe-33b6-470b-9a0f-f3ddcd70377c" path="/var/lib/kubelet/pods/220c07fe-33b6-470b-9a0f-f3ddcd70377c/volumes" Jul 24 13:54:43.204976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:43.204937 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:43.207450 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:54:43.207421 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46916001_5a2c_4be7_9169_c523e8ec418f.slice/crio-b609a24fd12c8e62cbde285c6852e19a6989ec222d70f63e8cf9a53b2a242a1b WatchSource:0}: Error finding container b609a24fd12c8e62cbde285c6852e19a6989ec222d70f63e8cf9a53b2a242a1b: Status 404 returned error can't find the container with id b609a24fd12c8e62cbde285c6852e19a6989ec222d70f63e8cf9a53b2a242a1b Jul 24 13:54:43.376540 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:43.376503 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825882224368" event={"ID":"46916001-5a2c-4be7-9169-c523e8ec418f","Type":"ContainerStarted","Data":"b609a24fd12c8e62cbde285c6852e19a6989ec222d70f63e8cf9a53b2a242a1b"} Jul 24 13:54:43.377734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:43.377708 2558 scope.go:117] "RemoveContainer" containerID="64b886ac38abd2b356162e3cfbf1506f7113547dd3c51cea3d676d9829148d92" Jul 24 13:54:43.377900 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:43.377734 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883264560" Jul 24 13:54:44.386648 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:44.386612 2558 generic.go:356] "Generic (PLEG): container finished" podID="46916001-5a2c-4be7-9169-c523e8ec418f" containerID="3c9198cf909840fab947dc8567788028e5da400bea70a8916268b4cb09fad425" exitCode=0 Jul 24 13:54:44.387074 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:44.386705 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825882224368" event={"ID":"46916001-5a2c-4be7-9169-c523e8ec418f","Type":"ContainerDied","Data":"3c9198cf909840fab947dc8567788028e5da400bea70a8916268b4cb09fad425"} Jul 24 13:54:45.544433 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.544372 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:45.564859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.564806 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7646-139825882224368_46916001-5a2c-4be7-9169-c523e8ec418f/test-curl-7646-139825882224368/0.log" Jul 24 13:54:45.588601 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.588551 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:45.592177 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.592137 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:45.601313 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.601277 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf97m\" (UniqueName: \"kubernetes.io/projected/46916001-5a2c-4be7-9169-c523e8ec418f-kube-api-access-kf97m\") pod \"46916001-5a2c-4be7-9169-c523e8ec418f\" (UID: \"46916001-5a2c-4be7-9169-c523e8ec418f\") " Jul 24 13:54:45.603413 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.603347 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46916001-5a2c-4be7-9169-c523e8ec418f-kube-api-access-kf97m" (OuterVolumeSpecName: "kube-api-access-kf97m") pod "46916001-5a2c-4be7-9169-c523e8ec418f" (UID: "46916001-5a2c-4be7-9169-c523e8ec418f"). InnerVolumeSpecName "kube-api-access-kf97m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:45.702890 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.702792 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf97m\" (UniqueName: \"kubernetes.io/projected/46916001-5a2c-4be7-9169-c523e8ec418f-kube-api-access-kf97m\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:54:45.739224 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.739184 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:45.739663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.739647 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46916001-5a2c-4be7-9169-c523e8ec418f" containerName="test-curl-7646-139825882224368" Jul 24 13:54:45.739713 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.739663 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="46916001-5a2c-4be7-9169-c523e8ec418f" containerName="test-curl-7646-139825882224368" Jul 24 13:54:45.739747 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.739730 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="46916001-5a2c-4be7-9169-c523e8ec418f" containerName="test-curl-7646-139825882224368" Jul 24 13:54:45.742816 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.742797 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:45.749026 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.748991 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:45.803374 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.803333 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lx8z\" (UniqueName: \"kubernetes.io/projected/9ae22540-a5d0-493e-b7f7-8ecd632212fc-kube-api-access-8lx8z\") pod \"test-curl-7646-139825882224368\" (UID: \"9ae22540-a5d0-493e-b7f7-8ecd632212fc\") " pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:45.903998 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.903961 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lx8z\" (UniqueName: \"kubernetes.io/projected/9ae22540-a5d0-493e-b7f7-8ecd632212fc-kube-api-access-8lx8z\") pod \"test-curl-7646-139825882224368\" (UID: \"9ae22540-a5d0-493e-b7f7-8ecd632212fc\") " pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:45.912130 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:45.912097 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lx8z\" (UniqueName: \"kubernetes.io/projected/9ae22540-a5d0-493e-b7f7-8ecd632212fc-kube-api-access-8lx8z\") pod \"test-curl-7646-139825882224368\" (UID: \"9ae22540-a5d0-493e-b7f7-8ecd632212fc\") " pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:46.054665 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.054577 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:46.197891 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.197721 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:46.200259 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:54:46.200217 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ae22540_a5d0_493e_b7f7_8ecd632212fc.slice/crio-90500f6d9f654633a50890b03016448b5b321e25f377cbe3862596afde721472 WatchSource:0}: Error finding container 90500f6d9f654633a50890b03016448b5b321e25f377cbe3862596afde721472: Status 404 returned error can't find the container with id 90500f6d9f654633a50890b03016448b5b321e25f377cbe3862596afde721472 Jul 24 13:54:46.213064 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.213019 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:54:46.399278 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.399237 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825882224368" event={"ID":"9ae22540-a5d0-493e-b7f7-8ecd632212fc","Type":"ContainerStarted","Data":"90500f6d9f654633a50890b03016448b5b321e25f377cbe3862596afde721472"} Jul 24 13:54:46.400590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.400567 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:46.400699 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.400573 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b609a24fd12c8e62cbde285c6852e19a6989ec222d70f63e8cf9a53b2a242a1b" Jul 24 13:54:46.415511 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:46.415469 2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7646-139825882224368" oldPodUID="46916001-5a2c-4be7-9169-c523e8ec418f" podUID="9ae22540-a5d0-493e-b7f7-8ecd632212fc" Jul 24 13:54:47.198726 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:47.198686 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46916001-5a2c-4be7-9169-c523e8ec418f" path="/var/lib/kubelet/pods/46916001-5a2c-4be7-9169-c523e8ec418f/volumes" Jul 24 13:54:47.408678 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:47.408637 2558 generic.go:356] "Generic (PLEG): container finished" podID="9ae22540-a5d0-493e-b7f7-8ecd632212fc" containerID="ffd7cac63eeb5b90f2766b588a69c6bce8ae4e93f56b386a2b373ecb6f356eb2" exitCode=0 Jul 24 13:54:47.408842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:47.408717 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825882224368" event={"ID":"9ae22540-a5d0-493e-b7f7-8ecd632212fc","Type":"ContainerDied","Data":"ffd7cac63eeb5b90f2766b588a69c6bce8ae4e93f56b386a2b373ecb6f356eb2"} Jul 24 13:54:48.574698 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.574664 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:48.592436 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.592402 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7646-139825882224368_9ae22540-a5d0-493e-b7f7-8ecd632212fc/test-curl-7646-139825882224368/0.log" Jul 24 13:54:48.616246 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.616193 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:48.618681 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.618645 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7646-139825882224368"] Jul 24 13:54:48.626926 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.626894 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lx8z\" (UniqueName: \"kubernetes.io/projected/9ae22540-a5d0-493e-b7f7-8ecd632212fc-kube-api-access-8lx8z\") pod \"9ae22540-a5d0-493e-b7f7-8ecd632212fc\" (UID: \"9ae22540-a5d0-493e-b7f7-8ecd632212fc\") " Jul 24 13:54:48.629097 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.629071 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae22540-a5d0-493e-b7f7-8ecd632212fc-kube-api-access-8lx8z" (OuterVolumeSpecName: "kube-api-access-8lx8z") pod "9ae22540-a5d0-493e-b7f7-8ecd632212fc" (UID: "9ae22540-a5d0-493e-b7f7-8ecd632212fc"). InnerVolumeSpecName "kube-api-access-8lx8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:48.728359 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.728265 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lx8z\" (UniqueName: \"kubernetes.io/projected/9ae22540-a5d0-493e-b7f7-8ecd632212fc-kube-api-access-8lx8z\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:54:48.753398 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.753356 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:48.753795 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.753780 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ae22540-a5d0-493e-b7f7-8ecd632212fc" containerName="test-curl-7646-139825882224368" Jul 24 13:54:48.753846 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.753795 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ae22540-a5d0-493e-b7f7-8ecd632212fc" containerName="test-curl-7646-139825882224368" Jul 24 13:54:48.753889 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.753879 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ae22540-a5d0-493e-b7f7-8ecd632212fc" containerName="test-curl-7646-139825882224368" Jul 24 13:54:48.756890 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.756857 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:48.764233 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.764196 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:48.829473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.829428 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48k67\" (UniqueName: \"kubernetes.io/projected/300b9f4b-a5d3-47a5-95f1-89d6788155bd-kube-api-access-48k67\") pod \"test-curl-7646-139825883272656\" (UID: \"300b9f4b-a5d3-47a5-95f1-89d6788155bd\") " pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:48.930331 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.930288 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48k67\" (UniqueName: \"kubernetes.io/projected/300b9f4b-a5d3-47a5-95f1-89d6788155bd-kube-api-access-48k67\") pod \"test-curl-7646-139825883272656\" (UID: \"300b9f4b-a5d3-47a5-95f1-89d6788155bd\") " pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:48.944836 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:48.944797 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48k67\" (UniqueName: \"kubernetes.io/projected/300b9f4b-a5d3-47a5-95f1-89d6788155bd-kube-api-access-48k67\") pod \"test-curl-7646-139825883272656\" (UID: \"300b9f4b-a5d3-47a5-95f1-89d6788155bd\") " pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:49.068451 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:49.068354 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:49.200596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:49.200565 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ae22540-a5d0-493e-b7f7-8ecd632212fc" path="/var/lib/kubelet/pods/9ae22540-a5d0-493e-b7f7-8ecd632212fc/volumes" Jul 24 13:54:49.216059 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:49.216007 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:49.219362 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:54:49.219331 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod300b9f4b_a5d3_47a5_95f1_89d6788155bd.slice/crio-2de4933dff6b9d22aabccf3fcf98d8906f4c30830c5cac815e50069e9affe7d1 WatchSource:0}: Error finding container 2de4933dff6b9d22aabccf3fcf98d8906f4c30830c5cac815e50069e9affe7d1: Status 404 returned error can't find the container with id 2de4933dff6b9d22aabccf3fcf98d8906f4c30830c5cac815e50069e9affe7d1 Jul 24 13:54:49.421880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:49.421852 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825882224368" Jul 24 13:54:49.422105 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:49.421861 2558 scope.go:117] "RemoveContainer" containerID="ffd7cac63eeb5b90f2766b588a69c6bce8ae4e93f56b386a2b373ecb6f356eb2" Jul 24 13:54:49.423220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:49.423189 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883272656" event={"ID":"300b9f4b-a5d3-47a5-95f1-89d6788155bd","Type":"ContainerStarted","Data":"2de4933dff6b9d22aabccf3fcf98d8906f4c30830c5cac815e50069e9affe7d1"} Jul 24 13:54:50.431665 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:50.431631 2558 generic.go:356] "Generic (PLEG): container finished" podID="300b9f4b-a5d3-47a5-95f1-89d6788155bd" containerID="50cddf80156043337050e2291c5068e81ec8270daec6699eff176665ad08d129" exitCode=0 Jul 24 13:54:50.432105 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:50.431715 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883272656" event={"ID":"300b9f4b-a5d3-47a5-95f1-89d6788155bd","Type":"ContainerDied","Data":"50cddf80156043337050e2291c5068e81ec8270daec6699eff176665ad08d129"} Jul 24 13:54:51.592501 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.592472 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:51.611179 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.611137 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7646-139825883272656_300b9f4b-a5d3-47a5-95f1-89d6788155bd/test-curl-7646-139825883272656/0.log" Jul 24 13:54:51.637838 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.637798 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:51.643038 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.642995 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:51.657728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.657681 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48k67\" (UniqueName: \"kubernetes.io/projected/300b9f4b-a5d3-47a5-95f1-89d6788155bd-kube-api-access-48k67\") pod \"300b9f4b-a5d3-47a5-95f1-89d6788155bd\" (UID: \"300b9f4b-a5d3-47a5-95f1-89d6788155bd\") " Jul 24 13:54:51.659807 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.659775 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/300b9f4b-a5d3-47a5-95f1-89d6788155bd-kube-api-access-48k67" (OuterVolumeSpecName: "kube-api-access-48k67") pod "300b9f4b-a5d3-47a5-95f1-89d6788155bd" (UID: "300b9f4b-a5d3-47a5-95f1-89d6788155bd"). InnerVolumeSpecName "kube-api-access-48k67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:51.758788 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.758693 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48k67\" (UniqueName: \"kubernetes.io/projected/300b9f4b-a5d3-47a5-95f1-89d6788155bd-kube-api-access-48k67\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:54:51.772066 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.772032 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:51.772464 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.772448 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="300b9f4b-a5d3-47a5-95f1-89d6788155bd" containerName="test-curl-7646-139825883272656" Jul 24 13:54:51.772536 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.772466 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="300b9f4b-a5d3-47a5-95f1-89d6788155bd" containerName="test-curl-7646-139825883272656" Jul 24 13:54:51.772573 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.772563 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="300b9f4b-a5d3-47a5-95f1-89d6788155bd" containerName="test-curl-7646-139825883272656" Jul 24 13:54:51.775757 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.775738 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:51.781106 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.781069 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:51.860081 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.860039 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grrbl\" (UniqueName: \"kubernetes.io/projected/2793b373-c3c4-4c88-9ccf-62daa11e44d6-kube-api-access-grrbl\") pod \"test-curl-7646-139825883272656\" (UID: \"2793b373-c3c4-4c88-9ccf-62daa11e44d6\") " pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:51.961590 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.961547 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grrbl\" (UniqueName: \"kubernetes.io/projected/2793b373-c3c4-4c88-9ccf-62daa11e44d6-kube-api-access-grrbl\") pod \"test-curl-7646-139825883272656\" (UID: \"2793b373-c3c4-4c88-9ccf-62daa11e44d6\") " pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:51.970155 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:51.970110 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grrbl\" (UniqueName: \"kubernetes.io/projected/2793b373-c3c4-4c88-9ccf-62daa11e44d6-kube-api-access-grrbl\") pod \"test-curl-7646-139825883272656\" (UID: \"2793b373-c3c4-4c88-9ccf-62daa11e44d6\") " pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:52.086597 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:52.086490 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:52.232475 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:52.232443 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:52.234520 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:54:52.234495 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2793b373_c3c4_4c88_9ccf_62daa11e44d6.slice/crio-78f92612f610a7f5d6352287375dfa54e2bba8873e2f93af9b21724c5ca00d89 WatchSource:0}: Error finding container 78f92612f610a7f5d6352287375dfa54e2bba8873e2f93af9b21724c5ca00d89: Status 404 returned error can't find the container with id 78f92612f610a7f5d6352287375dfa54e2bba8873e2f93af9b21724c5ca00d89 Jul 24 13:54:52.446552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:52.446520 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2de4933dff6b9d22aabccf3fcf98d8906f4c30830c5cac815e50069e9affe7d1" Jul 24 13:54:52.446552 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:52.446533 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:52.447827 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:52.447801 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883272656" event={"ID":"2793b373-c3c4-4c88-9ccf-62daa11e44d6","Type":"ContainerStarted","Data":"78f92612f610a7f5d6352287375dfa54e2bba8873e2f93af9b21724c5ca00d89"} Jul 24 13:54:52.458521 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:52.458482 2558 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7646-139825883272656" oldPodUID="300b9f4b-a5d3-47a5-95f1-89d6788155bd" podUID="2793b373-c3c4-4c88-9ccf-62daa11e44d6" Jul 24 13:54:53.197452 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:53.197416 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="300b9f4b-a5d3-47a5-95f1-89d6788155bd" path="/var/lib/kubelet/pods/300b9f4b-a5d3-47a5-95f1-89d6788155bd/volumes" Jul 24 13:54:53.458513 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:53.458411 2558 generic.go:356] "Generic (PLEG): container finished" podID="2793b373-c3c4-4c88-9ccf-62daa11e44d6" containerID="2f11e76c0e4fd16afd767524eb13eb25e69f6c8c7514abf4c2e721c8138fa458" exitCode=0 Jul 24 13:54:53.458513 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:53.458482 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7646-139825883272656" event={"ID":"2793b373-c3c4-4c88-9ccf-62daa11e44d6","Type":"ContainerDied","Data":"2f11e76c0e4fd16afd767524eb13eb25e69f6c8c7514abf4c2e721c8138fa458"} Jul 24 13:54:54.634287 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.634257 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:54.654320 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.654275 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7646-139825883272656_2793b373-c3c4-4c88-9ccf-62daa11e44d6/test-curl-7646-139825883272656/0.log" Jul 24 13:54:54.680976 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.680926 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:54.685756 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.685725 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7646-139825883272656"] Jul 24 13:54:54.687070 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.687042 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grrbl\" (UniqueName: \"kubernetes.io/projected/2793b373-c3c4-4c88-9ccf-62daa11e44d6-kube-api-access-grrbl\") pod \"2793b373-c3c4-4c88-9ccf-62daa11e44d6\" (UID: \"2793b373-c3c4-4c88-9ccf-62daa11e44d6\") " Jul 24 13:54:54.689141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.689114 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2793b373-c3c4-4c88-9ccf-62daa11e44d6-kube-api-access-grrbl" (OuterVolumeSpecName: "kube-api-access-grrbl") pod "2793b373-c3c4-4c88-9ccf-62daa11e44d6" (UID: "2793b373-c3c4-4c88-9ccf-62daa11e44d6"). InnerVolumeSpecName "kube-api-access-grrbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:54:54.788539 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:54.788443 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grrbl\" (UniqueName: \"kubernetes.io/projected/2793b373-c3c4-4c88-9ccf-62daa11e44d6-kube-api-access-grrbl\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:54:55.196711 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:55.196674 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2793b373-c3c4-4c88-9ccf-62daa11e44d6" path="/var/lib/kubelet/pods/2793b373-c3c4-4c88-9ccf-62daa11e44d6/volumes" Jul 24 13:54:55.236720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:55.236688 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:55.472077 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:55.471997 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7646-139825883272656" Jul 24 13:54:55.472077 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:55.472018 2558 scope.go:117] "RemoveContainer" containerID="2f11e76c0e4fd16afd767524eb13eb25e69f6c8c7514abf4c2e721c8138fa458" Jul 24 13:54:55.574512 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:55.574475 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:56.053357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:56.053315 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:58.679511 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:58.679461 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:58.767700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:58.767659 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:58.892044 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:58.892005 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:59.223339 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:59.223302 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:59.385969 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:59.385924 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:59.481663 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:54:59.481571 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:03.090175 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:03.090142 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:07.016567 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:07.016522 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:55:14.074814 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.074774 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:14.385351 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.385307 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4"] Jul 24 13:55:14.385816 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.385795 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2793b373-c3c4-4c88-9ccf-62daa11e44d6" containerName="test-curl-7646-139825883272656" Jul 24 13:55:14.385816 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.385814 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2793b373-c3c4-4c88-9ccf-62daa11e44d6" containerName="test-curl-7646-139825883272656" Jul 24 13:55:14.385991 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.385881 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="2793b373-c3c4-4c88-9ccf-62daa11e44d6" containerName="test-curl-7646-139825883272656" Jul 24 13:55:14.391471 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.391439 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.395539 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.395514 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 13:55:14.395702 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.395515 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-e01298-openshift-default-dockercfg-4qwjh\"" Jul 24 13:55:14.404490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.404455 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4"] Jul 24 13:55:14.430629 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.430590 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:14.576858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576817 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-data\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576864 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-credential-socket\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576898 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b3cbe6-8210-4475-b5a9-33f541811a67-istiod-ca-cert\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576917 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6ngl\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-kube-api-access-f6ngl\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576936 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-podinfo\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-socket\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.576979 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-certs\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577037 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.577003 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-envoy\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.577287 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.577067 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-token\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.677870 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.677748 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-credential-socket\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.677870 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.677827 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b3cbe6-8210-4475-b5a9-33f541811a67-istiod-ca-cert\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.677870 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.677857 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6ngl\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-kube-api-access-f6ngl\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.677962 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-podinfo\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678032 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-socket\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678063 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-certs\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678113 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-envoy\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678220 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678173 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-token\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678561 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678239 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-data\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678644 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678603 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-socket\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678747 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678722 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-credential-socket\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678762 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-data\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678766 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:14.678821 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678620 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-certs\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.678972 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.678842 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b3cbe6-8210-4475-b5a9-33f541811a67-istiod-ca-cert\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.680684 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.680655 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-podinfo\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.680813 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.680696 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-envoy\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.694450 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.694418 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-token\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.694724 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.694703 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6ngl\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-kube-api-access-f6ngl\") pod \"e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.704647 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.704609 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:14.866898 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:14.866859 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4"] Jul 24 13:55:14.870628 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:55:14.870598 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b3cbe6_8210_4475_b5a9_33f541811a67.slice/crio-b5b728f42896ce0da1a3c2ff87001c57e89fe5119d9b5ffb51a8f5dc5eb42ff9 WatchSource:0}: Error finding container b5b728f42896ce0da1a3c2ff87001c57e89fe5119d9b5ffb51a8f5dc5eb42ff9: Status 404 returned error can't find the container with id b5b728f42896ce0da1a3c2ff87001c57e89fe5119d9b5ffb51a8f5dc5eb42ff9 Jul 24 13:55:15.594967 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:15.594924 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" event={"ID":"09b3cbe6-8210-4475-b5a9-33f541811a67","Type":"ContainerStarted","Data":"b5b728f42896ce0da1a3c2ff87001c57e89fe5119d9b5ffb51a8f5dc5eb42ff9"} Jul 24 13:55:17.402130 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:17.402062 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:17.402427 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:17.402175 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:17.402427 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:17.402205 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:17.609569 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:17.609520 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" event={"ID":"09b3cbe6-8210-4475-b5a9-33f541811a67","Type":"ContainerStarted","Data":"1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d"} Jul 24 13:55:17.633174 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:17.633111 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" podStartSLOduration=1.104314122 podStartE2EDuration="3.633093319s" totalImagesPullingTime="2.528779197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:17.632352909 +0000 UTC m=+2780.996277002" watchObservedRunningTime="2026-07-24 13:55:17.633093319 +0000 UTC m=+2780.997017410" Jul 24 13:55:17.705645 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:17.705537 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:18.710896 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:18.710859 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:19.620082 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:19.619987 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:19.621353 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:19.621322 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:19.632103 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:19.632067 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:19.734776 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:19.734740 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:19.830975 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:19.830934 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:20.075412 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.075282 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:20.344548 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.344453 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:20.443517 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.443483 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:20.468910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.468865 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd"] Jul 24 13:55:20.472913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.472888 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.475249 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.475217 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e01298\"" Jul 24 13:55:20.482375 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.482340 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd"] Jul 24 13:55:20.536266 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.536227 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2"] Jul 24 13:55:20.538276 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.538248 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.538411 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.538359 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqjz7\" (UniqueName: \"kubernetes.io/projected/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-kube-api-access-mqjz7\") pod \"payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.539938 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.539913 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.542341 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.542318 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e01298-dockercfg-ql84n\"" Jul 24 13:55:20.552216 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.552179 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2"] Jul 24 13:55:20.639498 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.639453 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b8f8869-1261-4de3-aea5-f139c9945cba-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.639719 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.639632 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvx55\" (UniqueName: \"kubernetes.io/projected/1b8f8869-1261-4de3-aea5-f139c9945cba-kube-api-access-nvx55\") pod \"payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.639796 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.639733 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqjz7\" (UniqueName: \"kubernetes.io/projected/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-kube-api-access-mqjz7\") pod \"payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.639860 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.639813 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.640540 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.640522 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.650268 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.650225 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqjz7\" (UniqueName: \"kubernetes.io/projected/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-kube-api-access-mqjz7\") pod \"payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.740306 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.740265 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b8f8869-1261-4de3-aea5-f139c9945cba-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.740720 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.740338 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvx55\" (UniqueName: \"kubernetes.io/projected/1b8f8869-1261-4de3-aea5-f139c9945cba-kube-api-access-nvx55\") pod \"payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.741029 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.741006 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b8f8869-1261-4de3-aea5-f139c9945cba-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.747955 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.747926 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvx55\" (UniqueName: \"kubernetes.io/projected/1b8f8869-1261-4de3-aea5-f139c9945cba-kube-api-access-nvx55\") pod \"payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.785053 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.785014 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:20.851284 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.851247 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:20.951243 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:20.951211 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd"] Jul 24 13:55:20.953651 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:55:20.953616 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac9f173_f28f_4b5b_ac8a_9506a7f91bcb.slice/crio-c2c146b010424e19d9c3916830c956b866157f7128933c73298186c471cca008 WatchSource:0}: Error finding container c2c146b010424e19d9c3916830c956b866157f7128933c73298186c471cca008: Status 404 returned error can't find the container with id c2c146b010424e19d9c3916830c956b866157f7128933c73298186c471cca008 Jul 24 13:55:21.028104 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.027939 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2"] Jul 24 13:55:21.030656 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:55:21.030626 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8f8869_1261_4de3_aea5_f139c9945cba.slice/crio-7e6fa4a8c1481413b9c16f0b871d98c742e7cd37a0b8bfbf0b3baf39cd0d7794 WatchSource:0}: Error finding container 7e6fa4a8c1481413b9c16f0b871d98c742e7cd37a0b8bfbf0b3baf39cd0d7794: Status 404 returned error can't find the container with id 7e6fa4a8c1481413b9c16f0b871d98c742e7cd37a0b8bfbf0b3baf39cd0d7794 Jul 24 13:55:21.633324 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.633284 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" event={"ID":"1b8f8869-1261-4de3-aea5-f139c9945cba","Type":"ContainerStarted","Data":"d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20"} Jul 24 13:55:21.633577 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.633353 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:21.633577 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.633380 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" event={"ID":"1b8f8869-1261-4de3-aea5-f139c9945cba","Type":"ContainerStarted","Data":"7e6fa4a8c1481413b9c16f0b871d98c742e7cd37a0b8bfbf0b3baf39cd0d7794"} Jul 24 13:55:21.634913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.634888 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" event={"ID":"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb","Type":"ContainerStarted","Data":"dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00"} Jul 24 13:55:21.635031 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.634943 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" event={"ID":"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb","Type":"ContainerStarted","Data":"c2c146b010424e19d9c3916830c956b866157f7128933c73298186c471cca008"} Jul 24 13:55:21.635098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.635062 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:21.655194 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.655136 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" podStartSLOduration=1.655120235 podStartE2EDuration="1.655120235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:21.653676068 +0000 UTC m=+2785.017600159" watchObservedRunningTime="2026-07-24 13:55:21.655120235 +0000 UTC m=+2785.019044325" Jul 24 13:55:21.670467 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:21.670381 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" podStartSLOduration=1.670363526 podStartE2EDuration="1.670363526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:21.669821744 +0000 UTC m=+2785.033745836" watchObservedRunningTime="2026-07-24 13:55:21.670363526 +0000 UTC m=+2785.034287617" Jul 24 13:55:24.202752 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:24.202694 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:26.419264 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:26.419219 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:55:32.643042 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:32.643002 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:55:32.643515 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:32.643087 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:55:34.694254 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.693734 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": read tcp 10.132.0.2:54696->10.132.0.71:15021: read: connection reset by peer" start-of-body= Jul 24 13:55:34.694254 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.693830 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": read tcp 10.132.0.2:54696->10.132.0.71:15021: read: connection reset by peer" Jul 24 13:55:34.722524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.722488 2558 generic.go:356] "Generic (PLEG): container finished" podID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerID="1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d" exitCode=0 Jul 24 13:55:34.722714 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.722530 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" event={"ID":"09b3cbe6-8210-4475-b5a9-33f541811a67","Type":"ContainerDied","Data":"1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d"} Jul 24 13:55:34.722977 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.722951 2558 scope.go:117] "RemoveContainer" containerID="1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d" Jul 24 13:55:34.724117 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.724084 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:34.724194 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.724157 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:34.724233 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:34.724195 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:35.730003 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:35.729964 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" event={"ID":"09b3cbe6-8210-4475-b5a9-33f541811a67","Type":"ContainerStarted","Data":"8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca"} Jul 24 13:55:36.705148 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:36.705105 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:36.705358 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:36.705333 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 24 13:55:36.705445 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:36.705420 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 24 13:55:37.705661 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:37.705623 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 24 13:55:37.706091 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:37.705691 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 24 13:55:38.705185 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:38.705097 2558 patch_prober.go:28] interesting pod/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 24 13:55:38.705185 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:38.705175 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 24 13:55:39.709642 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:39.709608 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:39.710116 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:39.710013 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:39.710765 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:39.710747 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:55:42.049208 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:55:42.049176 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd_1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb/payload-processing/0.log" Jul 24 13:56:07.955589 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:07.955548 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:12.422007 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:12.421967 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:56:28.512761 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:28.512702 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:56:39.687948 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:39.687914 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd_1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb/payload-processing/0.log" Jul 24 13:56:39.994696 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:39.994597 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:40.330000 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.329889 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf"] Jul 24 13:56:40.333909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.333882 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.336534 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.336511 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-095476-openshift-default-dockercfg-x5jm2\"" Jul 24 13:56:40.346465 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.346434 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf"] Jul 24 13:56:40.359161 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.359123 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:40.415692 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415646 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.415884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415711 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-socket\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.415884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415745 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-credential-socket\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.415884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415763 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-podinfo\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.415884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415788 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-certs\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.415884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415863 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-token\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.416052 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415908 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-data\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.416052 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415927 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-envoy\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.416052 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.415943 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s7n9\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-kube-api-access-4s7n9\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517435 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517364 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517635 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517466 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-socket\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517635 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517489 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-credential-socket\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517635 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517511 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-podinfo\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517635 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517553 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-certs\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517635 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517596 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-token\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517638 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-data\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517680 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-envoy\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.517858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517707 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s7n9\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-kube-api-access-4s7n9\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.518004 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.517966 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-socket\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.518165 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.518137 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.518324 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.518300 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-certs\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.518439 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.518191 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-data\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.518518 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.518447 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-credential-socket\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.520323 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.520298 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-envoy\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.520611 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.520591 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-podinfo\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.526992 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.526957 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s7n9\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-kube-api-access-4s7n9\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.526992 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.526989 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-token\") pod \"e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.648587 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.648545 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:40.672248 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.672208 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:40.807312 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.807281 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf"] Jul 24 13:56:40.809451 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:56:40.809417 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79344ec4_9807_4e5a_b54a_c5b67dbe472e.slice/crio-7e6fda212f5913cf6a8c0380a779f2104c808bfe25de777a7b899905a1b0a45b WatchSource:0}: Error finding container 7e6fda212f5913cf6a8c0380a779f2104c808bfe25de777a7b899905a1b0a45b: Status 404 returned error can't find the container with id 7e6fda212f5913cf6a8c0380a779f2104c808bfe25de777a7b899905a1b0a45b Jul 24 13:56:40.811959 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.811921 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:56:40.812044 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.812002 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:56:40.812092 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:40.812044 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:56:41.098719 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:41.098666 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" event={"ID":"79344ec4-9807-4e5a-b54a-c5b67dbe472e","Type":"ContainerStarted","Data":"df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98"} Jul 24 13:56:41.098924 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:41.098749 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" event={"ID":"79344ec4-9807-4e5a-b54a-c5b67dbe472e","Type":"ContainerStarted","Data":"7e6fda212f5913cf6a8c0380a779f2104c808bfe25de777a7b899905a1b0a45b"} Jul 24 13:56:41.118363 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:41.118303 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podStartSLOduration=1.118281747 podStartE2EDuration="1.118281747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:41.116321241 +0000 UTC m=+2864.480245333" watchObservedRunningTime="2026-07-24 13:56:41.118281747 +0000 UTC m=+2864.482205838" Jul 24 13:56:41.649085 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:41.649042 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:41.654416 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:41.654365 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:42.104700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:42.104666 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:42.105620 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:42.105600 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:56:43.149822 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:43.149783 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:43.237292 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:43.237259 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:43.374490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:43.374445 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:43.670696 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:43.670655 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:43.907805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:43.907753 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:44.008914 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:44.008818 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:47.676438 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:47.676403 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:50.815093 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:50.815000 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:56:58.145608 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:58.145555 2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 13:56:58.146011 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:58.145635 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 13:56:59.214269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:59.214169 2558 generic.go:356] "Generic (PLEG): container finished" podID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerID="df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98" exitCode=0 Jul 24 13:56:59.214269 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:59.214244 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" event={"ID":"79344ec4-9807-4e5a-b54a-c5b67dbe472e","Type":"ContainerDied","Data":"df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98"} Jul 24 13:56:59.214885 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:59.214815 2558 scope.go:117] "RemoveContainer" containerID="df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98" Jul 24 13:56:59.216084 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:59.216053 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:56:59.216176 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:59.216125 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:56:59.216176 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:56:59.216158 2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:57:00.221989 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:00.221953 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" event={"ID":"79344ec4-9807-4e5a-b54a-c5b67dbe472e","Type":"ContainerStarted","Data":"ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1"} Jul 24 13:57:00.649664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:00.649622 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:57:00.649936 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:00.649890 2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:57:00.650008 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:00.649946 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 13:57:01.649430 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:01.649365 2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:57:01.649867 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:01.649455 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 13:57:02.649783 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:02.649742 2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:57:02.650202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:02.649808 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 13:57:03.649535 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:03.649496 2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:57:03.649758 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:03.649563 2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 13:57:04.653228 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:04.653192 2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:57:04.653635 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:04.653485 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:57:04.654353 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:04.654336 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:57:19.330010 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:19.329972 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:19.390360 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:19.390311 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:19.492158 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:19.492121 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:23.921531 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:23.921493 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:57:24.097455 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:24.097422 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:24.537243 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:24.537209 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:30.718656 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:30.718615 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:30.821872 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:30.821833 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:31.006575 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:31.006459 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf"] Jul 24 13:57:31.006776 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:31.006749 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" containerID="cri-o://ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1" gracePeriod=30 Jul 24 13:57:31.023251 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:31.023216 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:31.294241 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:31.294146 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:34.656700 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:34.656645 2558 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 13:57:34.657088 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:34.656718 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:57:36.275312 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.275244 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:57:36.375258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375217 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-token\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375261 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-credential-socket\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375284 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-certs\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375329 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s7n9\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-kube-api-access-4s7n9\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375371 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-podinfo\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375455 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-envoy\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375499 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-data\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375553 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375537 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-socket\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375565 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istiod-ca-cert\") pod \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\" (UID: \"79344ec4-9807-4e5a-b54a-c5b67dbe472e\") " Jul 24 13:57:36.375880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375605 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:57:36.375880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375709 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:57:36.375880 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375822 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:57:36.376101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375938 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-socket\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.376101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375956 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-credential-socket\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.376101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.375971 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-workload-certs\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.376101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.376048 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:36.376561 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.376527 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-data" (OuterVolumeSpecName: "istio-data") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:57:36.377804 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.377778 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-token" (OuterVolumeSpecName: "istio-token") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:36.377922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.377856 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-kube-api-access-4s7n9" (OuterVolumeSpecName: "kube-api-access-4s7n9") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "kube-api-access-4s7n9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:36.377922 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.377849 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:57:36.378013 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.377984 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79344ec4-9807-4e5a-b54a-c5b67dbe472e" (UID: "79344ec4-9807-4e5a-b54a-c5b67dbe472e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:57:36.430796 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.430762 2558 generic.go:356] "Generic (PLEG): container finished" podID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerID="ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1" exitCode=0 Jul 24 13:57:36.430973 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.430817 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" Jul 24 13:57:36.430973 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.430839 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" event={"ID":"79344ec4-9807-4e5a-b54a-c5b67dbe472e","Type":"ContainerDied","Data":"ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1"} Jul 24 13:57:36.430973 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.430904 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf" event={"ID":"79344ec4-9807-4e5a-b54a-c5b67dbe472e","Type":"ContainerDied","Data":"7e6fda212f5913cf6a8c0380a779f2104c808bfe25de777a7b899905a1b0a45b"} Jul 24 13:57:36.430973 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.430936 2558 scope.go:117] "RemoveContainer" containerID="ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1" Jul 24 13:57:36.441098 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.441074 2558 scope.go:117] "RemoveContainer" containerID="df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98" Jul 24 13:57:36.450271 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.450250 2558 scope.go:117] "RemoveContainer" containerID="ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1" Jul 24 13:57:36.450615 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:57:36.450589 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1\": container with ID starting with ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1 not found: ID does not exist" containerID="ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1" Jul 24 13:57:36.450712 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.450622 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1"} err="failed to get container status \"ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1\": rpc error: code = NotFound desc = could not find container \"ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1\": container with ID starting with ed927e8b249ca252518fbb220820216d75430be675d6d0ed71268d5f6a62ddf1 not found: ID does not exist" Jul 24 13:57:36.450712 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.450641 2558 scope.go:117] "RemoveContainer" containerID="df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98" Jul 24 13:57:36.450855 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:57:36.450840 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98\": container with ID starting with df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98 not found: ID does not exist" containerID="df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98" Jul 24 13:57:36.450897 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.450859 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98"} err="failed to get container status \"df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98\": rpc error: code = NotFound desc = could not find container \"df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98\": container with ID starting with df0347bc2f25899f0243208dd41744dd76af7d579e480a8f80627279ffd2be98 not found: ID does not exist" Jul 24 13:57:36.463889 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.463849 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf"] Jul 24 13:57:36.471763 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.471730 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-095476-openshift-default-77759c9654-ldzxf"] Jul 24 13:57:36.476712 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.476684 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-token\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.476712 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.476714 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s7n9\" (UniqueName: \"kubernetes.io/projected/79344ec4-9807-4e5a-b54a-c5b67dbe472e-kube-api-access-4s7n9\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.476859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.476724 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-podinfo\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.476859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.476735 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-envoy\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.476859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.476744 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istio-data\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:36.476859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:36.476752 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79344ec4-9807-4e5a-b54a-c5b67dbe472e-istiod-ca-cert\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:57:37.198728 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:37.198691 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" path="/var/lib/kubelet/pods/79344ec4-9807-4e5a-b54a-c5b67dbe472e/volumes" Jul 24 13:57:50.413401 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:50.413355 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:57:50.980108 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:50.980075 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:51.089463 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:51.089423 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:51.191079 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:51.191041 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:51.270593 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:51.270492 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:55.202697 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:55.202658 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:55.681173 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:57:55.681137 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:00.705434 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:00.705398 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:02.318975 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:02.318933 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:58:37.351750 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:37.351711 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:37.500258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:37.500216 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:43.072879 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:43.072835 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:49.408511 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:49.408471 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:57.124058 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:57.124017 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:57.186725 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:57.186687 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:57.331536 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:57.331508 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:58:57.346734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:57.346700 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 13:58:57.435805 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:57.435669 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:57.688094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:58:57.688000 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:08.404225 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.404174 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:08.424893 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.424850 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd"] Jul 24 13:59:08.425292 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.425250 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" podUID="1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" containerName="payload-processing" containerID="cri-o://dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00" gracePeriod=30 Jul 24 13:59:08.433722 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.433557 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2"] Jul 24 13:59:08.433858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.433813 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" podUID="1b8f8869-1261-4de3-aea5-f139c9945cba" containerName="payload-pre-processing" containerID="cri-o://d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20" gracePeriod=30 Jul 24 13:59:08.503103 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.503069 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:08.561813 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.561730 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:08.733067 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.733028 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:59:08.736150 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.736124 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:59:08.868308 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.868270 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-plugins-config-volume\") pod \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " Jul 24 13:59:08.868308 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.868315 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqjz7\" (UniqueName: \"kubernetes.io/projected/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-kube-api-access-mqjz7\") pod \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\" (UID: \"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb\") " Jul 24 13:59:08.868591 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.868344 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b8f8869-1261-4de3-aea5-f139c9945cba-plugins-config-volume\") pod \"1b8f8869-1261-4de3-aea5-f139c9945cba\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " Jul 24 13:59:08.868591 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.868462 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvx55\" (UniqueName: \"kubernetes.io/projected/1b8f8869-1261-4de3-aea5-f139c9945cba-kube-api-access-nvx55\") pod \"1b8f8869-1261-4de3-aea5-f139c9945cba\" (UID: \"1b8f8869-1261-4de3-aea5-f139c9945cba\") " Jul 24 13:59:08.868758 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.868731 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" (UID: "1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:08.868903 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.868866 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b8f8869-1261-4de3-aea5-f139c9945cba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b8f8869-1261-4de3-aea5-f139c9945cba" (UID: "1b8f8869-1261-4de3-aea5-f139c9945cba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:08.870632 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.870600 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b8f8869-1261-4de3-aea5-f139c9945cba-kube-api-access-nvx55" (OuterVolumeSpecName: "kube-api-access-nvx55") pod "1b8f8869-1261-4de3-aea5-f139c9945cba" (UID: "1b8f8869-1261-4de3-aea5-f139c9945cba"). InnerVolumeSpecName "kube-api-access-nvx55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:08.870782 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.870760 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-kube-api-access-mqjz7" (OuterVolumeSpecName: "kube-api-access-mqjz7") pod "1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" (UID: "1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb"). InnerVolumeSpecName "kube-api-access-mqjz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:08.942761 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.942726 2558 generic.go:356] "Generic (PLEG): container finished" podID="1b8f8869-1261-4de3-aea5-f139c9945cba" containerID="d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20" exitCode=0 Jul 24 13:59:08.942979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.942773 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" Jul 24 13:59:08.942979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.942815 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" event={"ID":"1b8f8869-1261-4de3-aea5-f139c9945cba","Type":"ContainerDied","Data":"d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20"} Jul 24 13:59:08.942979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.942874 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2" event={"ID":"1b8f8869-1261-4de3-aea5-f139c9945cba","Type":"ContainerDied","Data":"7e6fa4a8c1481413b9c16f0b871d98c742e7cd37a0b8bfbf0b3baf39cd0d7794"} Jul 24 13:59:08.942979 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.942901 2558 scope.go:117] "RemoveContainer" containerID="d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20" Jul 24 13:59:08.944232 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.944204 2558 generic.go:356] "Generic (PLEG): container finished" podID="1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" containerID="dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00" exitCode=0 Jul 24 13:59:08.944325 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.944254 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" Jul 24 13:59:08.944325 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.944258 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" event={"ID":"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb","Type":"ContainerDied","Data":"dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00"} Jul 24 13:59:08.944325 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.944308 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd" event={"ID":"1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb","Type":"ContainerDied","Data":"c2c146b010424e19d9c3916830c956b866157f7128933c73298186c471cca008"} Jul 24 13:59:08.956051 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.955984 2558 scope.go:117] "RemoveContainer" containerID="d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20" Jul 24 13:59:08.956734 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:08.956562 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20\": container with ID starting with d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20 not found: ID does not exist" containerID="d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20" Jul 24 13:59:08.956734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.956604 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20"} err="failed to get container status \"d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20\": rpc error: code = NotFound desc = could not find container \"d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20\": container with ID starting with d7f641750be8b9dc84a0ab99851bd5a914ee4d73ae6af3202f5959771771eb20 not found: ID does not exist" Jul 24 13:59:08.956734 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.956631 2558 scope.go:117] "RemoveContainer" containerID="dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00" Jul 24 13:59:08.967616 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.967566 2558 scope.go:117] "RemoveContainer" containerID="dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00" Jul 24 13:59:08.968030 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:08.968003 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00\": container with ID starting with dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00 not found: ID does not exist" containerID="dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00" Jul 24 13:59:08.968130 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.968041 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00"} err="failed to get container status \"dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00\": rpc error: code = NotFound desc = could not find container \"dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00\": container with ID starting with dcc544ffa8bb786bdc9ba48a5c313d9b894ea15cda96374c48a64373df09ee00 not found: ID does not exist" Jul 24 13:59:08.969650 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.969627 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvx55\" (UniqueName: \"kubernetes.io/projected/1b8f8869-1261-4de3-aea5-f139c9945cba-kube-api-access-nvx55\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.969650 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.969651 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.969771 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.969662 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqjz7\" (UniqueName: \"kubernetes.io/projected/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb-kube-api-access-mqjz7\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.969771 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.969673 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b8f8869-1261-4de3-aea5-f139c9945cba-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.970672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.970652 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd"] Jul 24 13:59:08.974020 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.973988 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e01298-5b7bd6f694-zf7wd"] Jul 24 13:59:08.984144 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.984105 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2"] Jul 24 13:59:08.986113 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:08.986088 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e01298-769b845448-dhcl2"] Jul 24 13:59:09.197852 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:09.197815 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" path="/var/lib/kubelet/pods/1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb/volumes" Jul 24 13:59:09.198188 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:09.198175 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b8f8869-1261-4de3-aea5-f139c9945cba" path="/var/lib/kubelet/pods/1b8f8869-1261-4de3-aea5-f139c9945cba/volumes" Jul 24 13:59:11.714566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:11.714524 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:59:12.917418 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:12.917365 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:13.297258 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:13.297169 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:13.397788 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:13.397742 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:18.944259 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:18.944221 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:19.005672 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:19.005633 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:19.204858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:19.204753 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4"] Jul 24 13:59:19.205054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:19.205034 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" containerID="cri-o://8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca" gracePeriod=30 Jul 24 13:59:19.220305 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:19.220267 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:19.471177 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:19.471081 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:24.486250 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.486177 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:59:24.624737 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624697 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6ngl\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-kube-api-access-f6ngl\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.624946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624765 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b3cbe6-8210-4475-b5a9-33f541811a67-istiod-ca-cert\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.624946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624830 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-data\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.624946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624847 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-socket\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.624946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624870 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-certs\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.624946 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624913 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-credential-socket\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.625214 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624947 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-podinfo\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.625214 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.624992 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-token\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.625214 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.625023 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-envoy\") pod \"09b3cbe6-8210-4475-b5a9-33f541811a67\" (UID: \"09b3cbe6-8210-4475-b5a9-33f541811a67\") " Jul 24 13:59:24.625473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.625272 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:24.625473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.625324 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:24.625473 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.625435 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09b3cbe6-8210-4475-b5a9-33f541811a67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:24.625641 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.625615 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:24.625859 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.625834 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-data" (OuterVolumeSpecName: "istio-data") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:24.627202 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.627174 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-kube-api-access-f6ngl" (OuterVolumeSpecName: "kube-api-access-f6ngl") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "kube-api-access-f6ngl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:24.627308 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.627293 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:59:24.627741 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.627722 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:24.627833 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.627773 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-token" (OuterVolumeSpecName: "istio-token") pod "09b3cbe6-8210-4475-b5a9-33f541811a67" (UID: "09b3cbe6-8210-4475-b5a9-33f541811a67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:24.726566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726526 2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-data\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726559 2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-socket\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726568 2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-workload-certs\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726566 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726578 2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-credential-socket\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726587 2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-podinfo\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726596 2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-token\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726604 2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09b3cbe6-8210-4475-b5a9-33f541811a67-istio-envoy\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726612 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6ngl\" (UniqueName: \"kubernetes.io/projected/09b3cbe6-8210-4475-b5a9-33f541811a67-kube-api-access-f6ngl\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:24.726843 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:24.726621 2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09b3cbe6-8210-4475-b5a9-33f541811a67-istiod-ca-cert\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:25.040875 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.040839 2558 generic.go:356] "Generic (PLEG): container finished" podID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerID="8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca" exitCode=0 Jul 24 13:59:25.041101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.040902 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" Jul 24 13:59:25.041101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.040933 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" event={"ID":"09b3cbe6-8210-4475-b5a9-33f541811a67","Type":"ContainerDied","Data":"8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca"} Jul 24 13:59:25.041101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.041027 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4" event={"ID":"09b3cbe6-8210-4475-b5a9-33f541811a67","Type":"ContainerDied","Data":"b5b728f42896ce0da1a3c2ff87001c57e89fe5119d9b5ffb51a8f5dc5eb42ff9"} Jul 24 13:59:25.041101 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.041060 2558 scope.go:117] "RemoveContainer" containerID="8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca" Jul 24 13:59:25.051644 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.051624 2558 scope.go:117] "RemoveContainer" containerID="1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d" Jul 24 13:59:25.060664 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.060641 2558 scope.go:117] "RemoveContainer" containerID="8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca" Jul 24 13:59:25.060985 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:25.060965 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca\": container with ID starting with 8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca not found: ID does not exist" containerID="8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca" Jul 24 13:59:25.061072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.060996 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca"} err="failed to get container status \"8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca\": rpc error: code = NotFound desc = could not find container \"8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca\": container with ID starting with 8f24661d2397c664b1d373818bdcf5eaa7898800f4e02ead0081d2f71aaa56ca not found: ID does not exist" Jul 24 13:59:25.061072 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.061016 2558 scope.go:117] "RemoveContainer" containerID="1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d" Jul 24 13:59:25.061460 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:25.061375 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d\": container with ID starting with 1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d not found: ID does not exist" containerID="1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d" Jul 24 13:59:25.061620 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.061585 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d"} err="failed to get container status \"1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d\": rpc error: code = NotFound desc = could not find container \"1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d\": container with ID starting with 1c889e71c220c86b8da38054237a3b2162bcb2dcaeb550c0b6acec697ef87f2d not found: ID does not exist" Jul 24 13:59:25.063863 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.063840 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4"] Jul 24 13:59:25.067883 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.067858 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-e01298-openshift-default-67b459d8fd-rjzq4"] Jul 24 13:59:25.196634 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:25.196600 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" path="/var/lib/kubelet/pods/09b3cbe6-8210-4475-b5a9-33f541811a67/volumes" Jul 24 13:59:30.190448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.190408 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:30.222766 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.222726 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg"] Jul 24 13:59:30.228792 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.228745 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt"] Jul 24 13:59:30.229094 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.229062 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" podUID="edc50b40-f120-41ec-a398-9e8b74ee7f1d" containerName="payload-pre-processing" containerID="cri-o://dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39" gracePeriod=30 Jul 24 13:59:30.291280 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.291241 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:30.344328 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.344280 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:30.374126 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:30.374088 2558 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-shared-a-934481: configmap "payload-processing-plugins-e2e-shared-a-934481" not found Jul 24 13:59:30.374306 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:30.374177 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume podName:85917efa-01af-4714-9f43-1da7aa37adb7 nodeName:}" failed. No retries permitted until 2026-07-24 13:59:30.874159013 +0000 UTC m=+3034.238083082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume") pod "payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" (UID: "85917efa-01af-4714-9f43-1da7aa37adb7") : configmap "payload-processing-plugins-e2e-shared-a-934481" not found Jul 24 13:59:30.503647 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.503620 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:59:30.676902 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.676863 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc50b40-f120-41ec-a398-9e8b74ee7f1d-plugins-config-volume\") pod \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " Jul 24 13:59:30.677126 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.676985 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bplbt\" (UniqueName: \"kubernetes.io/projected/edc50b40-f120-41ec-a398-9e8b74ee7f1d-kube-api-access-bplbt\") pod \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\" (UID: \"edc50b40-f120-41ec-a398-9e8b74ee7f1d\") " Jul 24 13:59:30.677406 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.677356 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edc50b40-f120-41ec-a398-9e8b74ee7f1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "edc50b40-f120-41ec-a398-9e8b74ee7f1d" (UID: "edc50b40-f120-41ec-a398-9e8b74ee7f1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:30.679131 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.679103 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc50b40-f120-41ec-a398-9e8b74ee7f1d-kube-api-access-bplbt" (OuterVolumeSpecName: "kube-api-access-bplbt") pod "edc50b40-f120-41ec-a398-9e8b74ee7f1d" (UID: "edc50b40-f120-41ec-a398-9e8b74ee7f1d"). InnerVolumeSpecName "kube-api-access-bplbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:30.777954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.777869 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edc50b40-f120-41ec-a398-9e8b74ee7f1d-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:30.777954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:30.777901 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bplbt\" (UniqueName: \"kubernetes.io/projected/edc50b40-f120-41ec-a398-9e8b74ee7f1d-kube-api-access-bplbt\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:30.878909 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:30.878870 2558 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-shared-a-934481: configmap "payload-processing-plugins-e2e-shared-a-934481" not found Jul 24 13:59:30.879088 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:30.878944 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume podName:85917efa-01af-4714-9f43-1da7aa37adb7 nodeName:}" failed. No retries permitted until 2026-07-24 13:59:31.878929459 +0000 UTC m=+3035.242853528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume") pod "payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" (UID: "85917efa-01af-4714-9f43-1da7aa37adb7") : configmap "payload-processing-plugins-e2e-shared-a-934481" not found Jul 24 13:59:31.078623 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.078533 2558 generic.go:356] "Generic (PLEG): container finished" podID="edc50b40-f120-41ec-a398-9e8b74ee7f1d" containerID="dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39" exitCode=0 Jul 24 13:59:31.078623 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.078580 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" Jul 24 13:59:31.078623 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.078614 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" event={"ID":"edc50b40-f120-41ec-a398-9e8b74ee7f1d","Type":"ContainerDied","Data":"dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39"} Jul 24 13:59:31.078884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.078669 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt" event={"ID":"edc50b40-f120-41ec-a398-9e8b74ee7f1d","Type":"ContainerDied","Data":"da8bf97023d0302b1d5978022a2d07d62efa99d72b2683f1b8ca32c0409f9602"} Jul 24 13:59:31.078884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.078695 2558 scope.go:117] "RemoveContainer" containerID="dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39" Jul 24 13:59:31.078884 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.078766 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" podUID="85917efa-01af-4714-9f43-1da7aa37adb7" containerName="payload-processing" containerID="cri-o://dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1" gracePeriod=30 Jul 24 13:59:31.089486 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.089464 2558 scope.go:117] "RemoveContainer" containerID="dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39" Jul 24 13:59:31.089792 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:31.089772 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39\": container with ID starting with dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39 not found: ID does not exist" containerID="dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39" Jul 24 13:59:31.089842 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.089801 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39"} err="failed to get container status \"dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39\": rpc error: code = NotFound desc = could not find container \"dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39\": container with ID starting with dfde799e204e8c09927ab0221149032a1c90ff067118a7d9c9664e21b040bb39 not found: ID does not exist" Jul 24 13:59:31.109217 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.109185 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt"] Jul 24 13:59:31.111057 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.111030 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-934481-848647db9b-95gvt"] Jul 24 13:59:31.197788 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.197756 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edc50b40-f120-41ec-a398-9e8b74ee7f1d" path="/var/lib/kubelet/pods/edc50b40-f120-41ec-a398-9e8b74ee7f1d/volumes" Jul 24 13:59:31.343054 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.342987 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:59:31.483448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.483407 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmqt9\" (UniqueName: \"kubernetes.io/projected/85917efa-01af-4714-9f43-1da7aa37adb7-kube-api-access-dmqt9\") pod \"85917efa-01af-4714-9f43-1da7aa37adb7\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " Jul 24 13:59:31.483448 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.483451 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume\") pod \"85917efa-01af-4714-9f43-1da7aa37adb7\" (UID: \"85917efa-01af-4714-9f43-1da7aa37adb7\") " Jul 24 13:59:31.483900 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.483874 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85917efa-01af-4714-9f43-1da7aa37adb7" (UID: "85917efa-01af-4714-9f43-1da7aa37adb7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:31.485668 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.485643 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85917efa-01af-4714-9f43-1da7aa37adb7-kube-api-access-dmqt9" (OuterVolumeSpecName: "kube-api-access-dmqt9") pod "85917efa-01af-4714-9f43-1da7aa37adb7" (UID: "85917efa-01af-4714-9f43-1da7aa37adb7"). InnerVolumeSpecName "kube-api-access-dmqt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:31.584936 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.584890 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmqt9\" (UniqueName: \"kubernetes.io/projected/85917efa-01af-4714-9f43-1da7aa37adb7-kube-api-access-dmqt9\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:31.584936 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:31.584928 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85917efa-01af-4714-9f43-1da7aa37adb7-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:32.085710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.085675 2558 generic.go:356] "Generic (PLEG): container finished" podID="85917efa-01af-4714-9f43-1da7aa37adb7" containerID="dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1" exitCode=0 Jul 24 13:59:32.085910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.085723 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" Jul 24 13:59:32.085910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.085768 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" event={"ID":"85917efa-01af-4714-9f43-1da7aa37adb7","Type":"ContainerDied","Data":"dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1"} Jul 24 13:59:32.085910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.085818 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" event={"ID":"85917efa-01af-4714-9f43-1da7aa37adb7","Type":"ContainerDied","Data":"6ab263e1001ee6ce6e241f004c0155ad6a093c6fe8d675e2ba5b7081191d0dc7"} Jul 24 13:59:32.085910 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.085841 2558 scope.go:117] "RemoveContainer" containerID="dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1" Jul 24 13:59:32.095150 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.095130 2558 scope.go:117] "RemoveContainer" containerID="dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1" Jul 24 13:59:32.095514 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:32.095488 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1\": container with ID starting with dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1 not found: ID does not exist" containerID="dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1" Jul 24 13:59:32.095596 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.095523 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1"} err="failed to get container status \"dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1\": rpc error: code = NotFound desc = could not find container \"dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1\": container with ID starting with dc1a3dd467d8d11d613ee98dd77ce0c777fb4561560d380439bc1e2f120889a1 not found: ID does not exist" Jul 24 13:59:32.107355 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.107323 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg"] Jul 24 13:59:32.111103 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.111075 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg"] Jul 24 13:59:32.303942 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.303890 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-934481-57f65df7f4-wqcvg" podUID="85917efa-01af-4714-9f43-1da7aa37adb7" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.63:9004: i/o timeout" Jul 24 13:59:32.713883 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:32.713841 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:59:33.196997 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:33.196953 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85917efa-01af-4714-9f43-1da7aa37adb7" path="/var/lib/kubelet/pods/85917efa-01af-4714-9f43-1da7aa37adb7/volumes" Jul 24 13:59:34.701247 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:34.701204 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:35.130524 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:35.130484 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:40.897476 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:40.897436 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:40.975447 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:40.975406 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:41.185687 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.185582 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:41.439650 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.439545 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:41.985357 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985320 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn"] Jul 24 13:59:41.985777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985748 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" Jul 24 13:59:41.985777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985762 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" Jul 24 13:59:41.985777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985770 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" Jul 24 13:59:41.985777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985776 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985788 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b8f8869-1261-4de3-aea5-f139c9945cba" containerName="payload-pre-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985794 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b8f8869-1261-4de3-aea5-f139c9945cba" containerName="payload-pre-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985810 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85917efa-01af-4714-9f43-1da7aa37adb7" containerName="payload-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985815 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="85917efa-01af-4714-9f43-1da7aa37adb7" containerName="payload-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985820 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" containerName="payload-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985825 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" containerName="payload-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985833 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc50b40-f120-41ec-a398-9e8b74ee7f1d" containerName="payload-pre-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985838 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc50b40-f120-41ec-a398-9e8b74ee7f1d" containerName="payload-pre-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985896 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="85917efa-01af-4714-9f43-1da7aa37adb7" containerName="payload-processing" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985903 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985909 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" Jul 24 13:59:41.985913 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985915 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac9f173-f28f-4b5b-ac8a-9506a7f91bcb" containerName="payload-processing" Jul 24 13:59:41.986309 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985922 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" Jul 24 13:59:41.986309 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985929 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc50b40-f120-41ec-a398-9e8b74ee7f1d" containerName="payload-pre-processing" Jul 24 13:59:41.986309 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.985937 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b8f8869-1261-4de3-aea5-f139c9945cba" containerName="payload-pre-processing" Jul 24 13:59:41.989003 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.988983 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:41.991787 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.991759 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j482l\"" Jul 24 13:59:41.991953 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.991761 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 13:59:41.997227 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:41.996782 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn"] Jul 24 13:59:42.084195 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:42.084151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/127ef7b3-3b17-4697-99e0-d8a5a35d1634-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn\" (UID: \"127ef7b3-3b17-4697-99e0-d8a5a35d1634\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:42.184956 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:42.184916 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/127ef7b3-3b17-4697-99e0-d8a5a35d1634-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn\" (UID: \"127ef7b3-3b17-4697-99e0-d8a5a35d1634\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:42.185639 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:42.185617 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/127ef7b3-3b17-4697-99e0-d8a5a35d1634-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn\" (UID: \"127ef7b3-3b17-4697-99e0-d8a5a35d1634\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:42.301630 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:42.301531 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:42.453710 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:42.453640 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn"] Jul 24 13:59:42.456268 ip-10-0-139-210 kubenswrapper[2558]: W0724 13:59:42.456228 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127ef7b3_3b17_4697_99e0_d8a5a35d1634.slice/crio-3f100de440a3605e620db8eb3c887930864c37439830cda5dcf152071427be92 WatchSource:0}: Error finding container 3f100de440a3605e620db8eb3c887930864c37439830cda5dcf152071427be92: Status 404 returned error can't find the container with id 3f100de440a3605e620db8eb3c887930864c37439830cda5dcf152071427be92 Jul 24 13:59:43.151379 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:43.151339 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" event={"ID":"127ef7b3-3b17-4697-99e0-d8a5a35d1634","Type":"ContainerStarted","Data":"a586734adf8ce76f941678635831f3ea1744efa725a01e18fdb6ef3b059075c4"} Jul 24 13:59:43.151815 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:43.151424 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" event={"ID":"127ef7b3-3b17-4697-99e0-d8a5a35d1634","Type":"ContainerStarted","Data":"3f100de440a3605e620db8eb3c887930864c37439830cda5dcf152071427be92"} Jul 24 13:59:43.170907 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:43.170851 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" podStartSLOduration=2.170832006 podStartE2EDuration="2.170832006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:59:43.170670848 +0000 UTC m=+3046.534594939" watchObservedRunningTime="2026-07-24 13:59:43.170832006 +0000 UTC m=+3046.534756075" Jul 24 13:59:44.158315 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:44.158276 2558 generic.go:356] "Generic (PLEG): container finished" podID="127ef7b3-3b17-4697-99e0-d8a5a35d1634" containerID="a586734adf8ce76f941678635831f3ea1744efa725a01e18fdb6ef3b059075c4" exitCode=0 Jul 24 13:59:44.158848 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:44.158364 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" event={"ID":"127ef7b3-3b17-4697-99e0-d8a5a35d1634","Type":"ContainerDied","Data":"a586734adf8ce76f941678635831f3ea1744efa725a01e18fdb6ef3b059075c4"} Jul 24 13:59:45.312766 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:45.312738 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:45.413909 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:45.413866 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/127ef7b3-3b17-4697-99e0-d8a5a35d1634-maas-api-service-ca\") pod \"127ef7b3-3b17-4697-99e0-d8a5a35d1634\" (UID: \"127ef7b3-3b17-4697-99e0-d8a5a35d1634\") " Jul 24 13:59:45.414248 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:45.414222 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/127ef7b3-3b17-4697-99e0-d8a5a35d1634-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "127ef7b3-3b17-4697-99e0-d8a5a35d1634" (UID: "127ef7b3-3b17-4697-99e0-d8a5a35d1634"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:45.515380 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:45.515283 2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/127ef7b3-3b17-4697-99e0-d8a5a35d1634-maas-api-service-ca\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.174840 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:46.174806 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" event={"ID":"127ef7b3-3b17-4697-99e0-d8a5a35d1634","Type":"ContainerDied","Data":"3f100de440a3605e620db8eb3c887930864c37439830cda5dcf152071427be92"} Jul 24 13:59:46.175019 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:46.174844 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn" Jul 24 13:59:46.175019 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:46.174857 2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f100de440a3605e620db8eb3c887930864c37439830cda5dcf152071427be92" Jul 24 13:59:52.417659 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.417566 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:52.452285 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.451605 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6"] Jul 24 13:59:52.453501 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.452919 2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" podUID="65619dfc-6f77-4b5f-82a4-8f056d672f38" containerName="payload-pre-processing" containerID="cri-o://033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9" gracePeriod=30 Jul 24 13:59:52.481456 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.481416 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:52.545490 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.545445 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:52.729858 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.729826 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:59:52.782674 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.782637 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65619dfc-6f77-4b5f-82a4-8f056d672f38-plugins-config-volume\") pod \"65619dfc-6f77-4b5f-82a4-8f056d672f38\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " Jul 24 13:59:52.782674 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.782676 2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86tsp\" (UniqueName: \"kubernetes.io/projected/65619dfc-6f77-4b5f-82a4-8f056d672f38-kube-api-access-86tsp\") pod \"65619dfc-6f77-4b5f-82a4-8f056d672f38\" (UID: \"65619dfc-6f77-4b5f-82a4-8f056d672f38\") " Jul 24 13:59:52.783129 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.783097 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65619dfc-6f77-4b5f-82a4-8f056d672f38-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65619dfc-6f77-4b5f-82a4-8f056d672f38" (UID: "65619dfc-6f77-4b5f-82a4-8f056d672f38"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:52.784954 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.784924 2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65619dfc-6f77-4b5f-82a4-8f056d672f38-kube-api-access-86tsp" (OuterVolumeSpecName: "kube-api-access-86tsp") pod "65619dfc-6f77-4b5f-82a4-8f056d672f38" (UID: "65619dfc-6f77-4b5f-82a4-8f056d672f38"). InnerVolumeSpecName "kube-api-access-86tsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:52.883514 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.883469 2558 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65619dfc-6f77-4b5f-82a4-8f056d672f38-plugins-config-volume\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:52.883514 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:52.883506 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86tsp\" (UniqueName: \"kubernetes.io/projected/65619dfc-6f77-4b5f-82a4-8f056d672f38-kube-api-access-86tsp\") on node \"ip-10-0-139-210.ec2.internal\" DevicePath \"\"" Jul 24 13:59:53.213939 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.213908 2558 generic.go:356] "Generic (PLEG): container finished" podID="65619dfc-6f77-4b5f-82a4-8f056d672f38" containerID="033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9" exitCode=0 Jul 24 13:59:53.214093 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.213955 2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" Jul 24 13:59:53.214093 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.213985 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" event={"ID":"65619dfc-6f77-4b5f-82a4-8f056d672f38","Type":"ContainerDied","Data":"033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9"} Jul 24 13:59:53.214093 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.214038 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6" event={"ID":"65619dfc-6f77-4b5f-82a4-8f056d672f38","Type":"ContainerDied","Data":"78d3ca812f502263a8a26fcdadf8f041812b781e2db2440f20d53f318ae02e57"} Jul 24 13:59:53.214093 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.214060 2558 scope.go:117] "RemoveContainer" containerID="033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9" Jul 24 13:59:53.223510 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.223489 2558 scope.go:117] "RemoveContainer" containerID="033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9" Jul 24 13:59:53.223800 ip-10-0-139-210 kubenswrapper[2558]: E0724 13:59:53.223777 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9\": container with ID starting with 033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9 not found: ID does not exist" containerID="033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9" Jul 24 13:59:53.223874 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.223815 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9"} err="failed to get container status \"033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9\": rpc error: code = NotFound desc = could not find container \"033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9\": container with ID starting with 033d559dfddb8dc82e4173a70af741f8fa0a711bec8f62001ce99c2133950ea9 not found: ID does not exist" Jul 24 13:59:53.231532 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.231492 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6"] Jul 24 13:59:53.238232 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:53.238189 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-0a0df7-794478c548-j6tw6"] Jul 24 13:59:55.196777 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:55.196740 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65619dfc-6f77-4b5f-82a4-8f056d672f38" path="/var/lib/kubelet/pods/65619dfc-6f77-4b5f-82a4-8f056d672f38/volumes" Jul 24 13:59:55.610068 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:55.610019 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-zz7mw"] Jul 24 13:59:57.055141 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:57.055105 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:57.059887 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:57.059853 2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn"] Jul 24 13:59:57.063798 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:57.063768 2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-0a0df7-qqlbn"] Jul 24 13:59:57.196900 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:57.196868 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="127ef7b3-3b17-4697-99e0-d8a5a35d1634" path="/var/lib/kubelet/pods/127ef7b3-3b17-4697-99e0-d8a5a35d1634/volumes" Jul 24 13:59:57.455309 ip-10-0-139-210 kubenswrapper[2558]: I0724 13:59:57.455272 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:03.313493 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:03.313444 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:03.429103 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:03.429070 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:03.533477 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:03.533439 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:03.686774 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:03.686737 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:07.692134 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:07.692094 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:10.573538 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:10.573503 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9gwb7_b805c8df-6b44-456f-bddd-7e4aa2ac49f7/manager/0.log" Jul 24 14:00:10.716935 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:10.716903 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nh9xd_aac51a14-c7af-49fe-8b35-f31eb4d4de05/manager/0.log" Jul 24 14:00:10.980986 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:10.980960 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-75q6g_39e86268-e319-40ae-811a-0d3390f5d16e/server/0.log" Jul 24 14:00:11.112429 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:11.112376 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-png6s_8e2ffe89-f3f9-4588-9477-7e5609e5a6b4/manager/0.log" Jul 24 14:00:11.236989 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:11.236906 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-9vzwc_df9f9a58-f2fc-4692-a046-fa35407b5b8e/copy-manifests/0.log" Jul 24 14:00:11.243619 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:11.243584 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-9vzwc_df9f9a58-f2fc-4692-a046-fa35407b5b8e/manager/0.log" Jul 24 14:00:12.037608 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:12.037572 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748330-hvbmz_1838dcd8-9fba-47be-a692-86002cbb6811/cleanup/0.log" Jul 24 14:00:13.934501 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:13.934467 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xbgcb_5a6b3ff9-4e43-47e0-b518-d7f71fe75e30/manager/0.log" Jul 24 14:00:14.052917 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:14.052877 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7nd9g_e2227c62-61a4-4e1b-b1ad-98e911742c42/manager/0.log" Jul 24 14:00:14.306313 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:14.306225 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xwn24_5ad7204c-6003-4ce5-9d4a-395470a048fb/registry-server/0.log" Jul 24 14:00:14.440289 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:14.440254 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ckvts_4d25266b-2a4c-4834-99bb-488ba55828c3/manager/0.log" Jul 24 14:00:14.556582 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:14.556486 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zz7mw_7d6a8e07-7178-4469-b1c2-4706602e12f7/limitador/0.log" Jul 24 14:00:14.692960 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:14.692921 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-snksz_43ee53d1-c0c0-4237-aa70-605c9ec61d60/manager/0.log" Jul 24 14:00:14.937482 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:14.937438 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:15.322894 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:15.322805 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76dcd6cb6b-kf6p2_8839b621-a1fe-47d9-a70f-3ad9db2099d7/kube-auth-proxy/0.log" Jul 24 14:00:15.961401 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:15.961362 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b84687b8c-kmckn_6e76b0c5-9196-42ef-8515-86b4ec2d1f86/router/0.log" Jul 24 14:00:16.702740 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:16.702708 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f9xx8g_48ab9ba5-ab8b-475a-9771-9a2928b976de/main/0.log" Jul 24 14:00:22.806814 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:22.806775 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:22.878833 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:22.878803 2558 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:26.125678 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:26.125644 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8jsc6_4971787d-2c92-40a4-94d9-6ab28883e02b/global-pull-secret-syncer/0.log" Jul 24 14:00:26.217138 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:26.217103 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4nd5b_05df2885-2fde-48f1-aaf8-1a820a0d2f2a/konnectivity-agent/0.log" Jul 24 14:00:26.359880 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:26.359849 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-210.ec2.internal_cb3099a2ddbd5f01479c1918b5f9b91d/haproxy/0.log" Jul 24 14:00:31.709695 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:31.709665 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xbgcb_5a6b3ff9-4e43-47e0-b518-d7f71fe75e30/manager/0.log" Jul 24 14:00:31.737865 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:31.737822 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7nd9g_e2227c62-61a4-4e1b-b1ad-98e911742c42/manager/0.log" Jul 24 14:00:31.829209 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:31.829170 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xwn24_5ad7204c-6003-4ce5-9d4a-395470a048fb/registry-server/0.log" Jul 24 14:00:32.053030 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:32.052891 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ckvts_4d25266b-2a4c-4834-99bb-488ba55828c3/manager/0.log" Jul 24 14:00:32.076915 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:32.076886 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-zz7mw_7d6a8e07-7178-4469-b1c2-4706602e12f7/limitador/0.log" Jul 24 14:00:32.191638 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:32.191606 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-snksz_43ee53d1-c0c0-4237-aa70-605c9ec61d60/manager/0.log" Jul 24 14:00:34.161689 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:34.161650 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-sbzh4_9b3afa1c-c462-4ce1-891c-7412da5b4566/cluster-monitoring-operator/0.log" Jul 24 14:00:34.486038 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:34.485955 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xx2hc_8880d140-9846-4046-8580-c268d042b180/node-exporter/0.log" Jul 24 14:00:34.506455 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:34.506417 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xx2hc_8880d140-9846-4046-8580-c268d042b180/kube-rbac-proxy/0.log" Jul 24 14:00:34.527809 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:34.527773 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xx2hc_8880d140-9846-4046-8580-c268d042b180/init-textfile/0.log" Jul 24 14:00:35.510944 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.510903 2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt"] Jul 24 14:00:35.511346 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511313 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65619dfc-6f77-4b5f-82a4-8f056d672f38" containerName="payload-pre-processing" Jul 24 14:00:35.511346 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511326 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="65619dfc-6f77-4b5f-82a4-8f056d672f38" containerName="payload-pre-processing" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511351 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511357 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511365 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511371 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="79344ec4-9807-4e5a-b54a-c5b67dbe472e" containerName="istio-proxy" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511397 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127ef7b3-3b17-4697-99e0-d8a5a35d1634" containerName="revoke-keys" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511403 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="127ef7b3-3b17-4697-99e0-d8a5a35d1634" containerName="revoke-keys" Jul 24 14:00:35.511465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511463 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="65619dfc-6f77-4b5f-82a4-8f056d672f38" containerName="payload-pre-processing" Jul 24 14:00:35.511680 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511474 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="09b3cbe6-8210-4475-b5a9-33f541811a67" containerName="istio-proxy" Jul 24 14:00:35.511680 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.511482 2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="127ef7b3-3b17-4697-99e0-d8a5a35d1634" containerName="revoke-keys" Jul 24 14:00:35.515170 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.515145 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.517598 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.517572 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mgcsj\"/\"kube-root-ca.crt\"" Jul 24 14:00:35.517862 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.517845 2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mgcsj\"/\"openshift-service-ca.crt\"" Jul 24 14:00:35.518014 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.517997 2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mgcsj\"/\"default-dockercfg-twfrm\"" Jul 24 14:00:35.523226 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.522856 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt"] Jul 24 14:00:35.573105 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.573061 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-proc\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.573105 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.573101 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-lib-modules\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.573326 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.573129 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-podres\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.573326 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.573210 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-sys\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.573326 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.573245 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpgcf\" (UniqueName: \"kubernetes.io/projected/74776972-aaa8-4547-b4ac-012fd3160742-kube-api-access-zpgcf\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.674727 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674687 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-sys\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.674727 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674729 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpgcf\" (UniqueName: \"kubernetes.io/projected/74776972-aaa8-4547-b4ac-012fd3160742-kube-api-access-zpgcf\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.674951 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674825 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-sys\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.674951 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674824 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-proc\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.674951 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674947 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-lib-modules\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.675057 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674978 2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-podres\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.675057 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.674904 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-proc\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.675147 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.675134 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-podres\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.675183 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.675136 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74776972-aaa8-4547-b4ac-012fd3160742-lib-modules\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.682950 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.682913 2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpgcf\" (UniqueName: \"kubernetes.io/projected/74776972-aaa8-4547-b4ac-012fd3160742-kube-api-access-zpgcf\") pod \"perf-node-gather-daemonset-ph2wt\" (UID: \"74776972-aaa8-4547-b4ac-012fd3160742\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.827206 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.827102 2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:35.975997 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:35.975968 2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt"] Jul 24 14:00:35.978609 ip-10-0-139-210 kubenswrapper[2558]: W0724 14:00:35.978579 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod74776972_aaa8_4547_b4ac_012fd3160742.slice/crio-e664263bef8ec2aa16e8f4199f1654c8546a98736bab510b6b8a2d3d5f4c1194 WatchSource:0}: Error finding container e664263bef8ec2aa16e8f4199f1654c8546a98736bab510b6b8a2d3d5f4c1194: Status 404 returned error can't find the container with id e664263bef8ec2aa16e8f4199f1654c8546a98736bab510b6b8a2d3d5f4c1194 Jul 24 14:00:36.258333 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.258299 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-5vfs6_0825d55b-6343-411d-8a8c-9dcc1d46bf93/networking-console-plugin/0.log" Jul 24 14:00:36.447653 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.447619 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" event={"ID":"74776972-aaa8-4547-b4ac-012fd3160742","Type":"ContainerStarted","Data":"88124fc13c680e2ee6d45e7bba9ebcb9df9ea1d25ea8465385f870fb2bc2823e"} Jul 24 14:00:36.447855 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.447680 2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" event={"ID":"74776972-aaa8-4547-b4ac-012fd3160742","Type":"ContainerStarted","Data":"e664263bef8ec2aa16e8f4199f1654c8546a98736bab510b6b8a2d3d5f4c1194"} Jul 24 14:00:36.447855 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.447721 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:36.466271 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.466207 2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" podStartSLOduration=1.466188523 podStartE2EDuration="1.466188523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:00:36.464067491 +0000 UTC m=+3099.827991584" watchObservedRunningTime="2026-07-24 14:00:36.466188523 +0000 UTC m=+3099.830112613" Jul 24 14:00:36.837191 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.837154 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/1.log" Jul 24 14:00:36.847243 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:36.847208 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-4sl4t_279aea8c-817d-4897-b402-9da959d25728/console-operator/2.log" Jul 24 14:00:37.303251 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:37.303221 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cd96db8fd-wrl6p_484e29c8-ae08-490f-b48e-2eeb17311591/console/0.log" Jul 24 14:00:37.344205 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:37.344168 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-fv88n_731228c1-003d-4c8b-9f37-2caab3adf288/download-server/0.log" Jul 24 14:00:37.953249 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:37.953214 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-sr5ct_a7f709b4-e9e2-428f-9c9c-ef397ad87868/volume-data-source-validator/0.log" Jul 24 14:00:39.055846 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:39.055812 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z9vkb_b5db3487-f9e9-412c-a5b3-d4e661de0280/dns/0.log" Jul 24 14:00:39.081509 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:39.081479 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z9vkb_b5db3487-f9e9-412c-a5b3-d4e661de0280/kube-rbac-proxy/0.log" Jul 24 14:00:39.158029 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:39.157997 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zm4hb_077ed00b-9339-4528-948b-daf1bebab5fc/dns-node-resolver/0.log" Jul 24 14:00:39.720243 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:39.720201 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87xcs_595f165a-81a3-4662-9fb1-6e5e7c7dacb9/node-ca/0.log" Jul 24 14:00:40.889476 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:40.889437 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76dcd6cb6b-kf6p2_8839b621-a1fe-47d9-a70f-3ad9db2099d7/kube-auth-proxy/0.log" Jul 24 14:00:41.072240 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:41.072208 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b84687b8c-kmckn_6e76b0c5-9196-42ef-8515-86b4ec2d1f86/router/0.log" Jul 24 14:00:41.885195 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:41.885165 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nwrp8_2c393ace-cf46-4319-8dc3-548b7bf80ce0/serve-healthcheck-canary/0.log" Jul 24 14:00:42.413154 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:42.413111 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-69k7c_46b2036d-7e89-44e0-9b58-dd8a30691fd8/insights-operator/0.log" Jul 24 14:00:42.414574 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:42.414549 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-69k7c_46b2036d-7e89-44e0-9b58-dd8a30691fd8/insights-operator/1.log" Jul 24 14:00:42.440578 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:42.440549 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjt56_aa847532-3ffa-44e0-b42a-0968ac99aade/kube-rbac-proxy/0.log" Jul 24 14:00:42.463258 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:42.463232 2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-ph2wt" Jul 24 14:00:42.464775 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:42.464749 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjt56_aa847532-3ffa-44e0-b42a-0968ac99aade/exporter/0.log" Jul 24 14:00:42.491465 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:42.491419 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hjt56_aa847532-3ffa-44e0-b42a-0968ac99aade/extractor/0.log" Jul 24 14:00:45.085539 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:45.085448 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9gwb7_b805c8df-6b44-456f-bddd-7e4aa2ac49f7/manager/0.log" Jul 24 14:00:45.465166 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:45.465135 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nh9xd_aac51a14-c7af-49fe-8b35-f31eb4d4de05/manager/0.log" Jul 24 14:00:45.712182 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:45.712152 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-75q6g_39e86268-e319-40ae-811a-0d3390f5d16e/server/0.log" Jul 24 14:00:45.836244 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:45.836153 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-png6s_8e2ffe89-f3f9-4588-9477-7e5609e5a6b4/manager/0.log" Jul 24 14:00:45.862629 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:45.862596 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-9vzwc_df9f9a58-f2fc-4692-a046-fa35407b5b8e/manager/0.log" Jul 24 14:00:45.883985 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:45.883956 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-9vzwc_df9f9a58-f2fc-4692-a046-fa35407b5b8e/copy-manifests/0.log" Jul 24 14:00:46.211965 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:46.211913 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76dcd6cb6b-kf6p2_8839b621-a1fe-47d9-a70f-3ad9db2099d7/kube-auth-proxy/0.log" Jul 24 14:00:46.357345 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:46.357302 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b84687b8c-kmckn_6e76b0c5-9196-42ef-8515-86b4ec2d1f86/router/0.log" Jul 24 14:00:48.000253 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:48.000220 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76dcd6cb6b-kf6p2_8839b621-a1fe-47d9-a70f-3ad9db2099d7/kube-auth-proxy/0.log" Jul 24 14:00:48.134151 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:48.134120 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b84687b8c-kmckn_6e76b0c5-9196-42ef-8515-86b4ec2d1f86/router/0.log" Jul 24 14:00:52.256474 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:52.256440 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-6qjf4_5f1c92fc-37ef-4c0e-b52b-05a14cfac011/migrator/0.log" Jul 24 14:00:52.278845 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:52.278817 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-6qjf4_5f1c92fc-37ef-4c0e-b52b-05a14cfac011/graceful-termination/0.log" Jul 24 14:00:52.657481 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:52.657444 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh_13956c89-0f25-43f9-be04-ab203a8de1f9/kube-storage-version-migrator-operator/1.log" Jul 24 14:00:52.659533 ip-10-0-139-210 kubenswrapper[2558]: I0724 14:00:52.659490 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-vwxbh_13956c89-0f25-43f9-be04-ab203a8de1f9/kube-storage-version-migrator-operator/0.log"