Jul 24 01:27:57.022762 ip-10-0-138-219 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 24 01:27:57.022776 ip-10-0-138-219 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 24 01:27:57.022786 ip-10-0-138-219 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 24 01:27:57.023092 ip-10-0-138-219 systemd[1]: Failed to start Kubernetes Kubelet. Jul 24 01:28:06.247979 ip-10-0-138-219 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 24 01:28:06.247995 ip-10-0-138-219 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 400dc3b2ba634ddda1e5d02731660a58 -- Jul 24 01:29:50.376250 ip-10-0-138-219 systemd[1]: Starting Kubernetes Kubelet... Jul 24 01:29:50.721782 ip-10-0-138-219 systemd[1]: Started Kubernetes Kubelet. Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.662110 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665557 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665566 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665569 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665572 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665575 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665578 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665580 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 01:29:50.747233 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665583 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665585 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665588 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665590 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665593 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665600 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665603 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665606 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665609 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665611 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665614 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665616 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665619 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665621 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665624 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665626 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665629 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665631 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665636 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 01:29:50.749385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665640 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665643 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665647 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665651 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665654 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665656 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665659 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665662 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665665 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665667 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665670 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665672 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665675 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665677 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665679 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665682 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665684 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665687 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665689 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 01:29:50.752003 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665692 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665695 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665698 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665700 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665702 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665705 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665708 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665710 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665713 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665715 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665718 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665720 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665723 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665725 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665727 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665730 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665732 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665734 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665737 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 01:29:50.754156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665739 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665742 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665744 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665747 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665750 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665752 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665756 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665760 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665763 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665765 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665768 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665771 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665773 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665776 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665778 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665782 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665784 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665786 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665789 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665791 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 01:29:50.754964 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665794 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665796 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665799 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665801 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665804 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665806 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665809 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665811 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665813 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665816 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665818 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665820 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665823 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665825 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.665827 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666534 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666541 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666545 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666548 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666551 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 01:29:50.757156 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666554 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666557 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666560 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666564 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666566 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666570 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666573 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666575 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666578 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666580 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666583 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666585 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666588 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666591 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666593 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666596 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666598 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666601 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666603 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666605 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 01:29:50.759085 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666608 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666610 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666613 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666615 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666618 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666621 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666624 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666627 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666629 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666631 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666634 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666637 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666639 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666642 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666644 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666648 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666652 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666655 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666659 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 01:29:50.760508 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666661 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666664 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666666 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666669 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666671 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666674 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666676 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666679 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666681 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666683 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666686 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666688 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666691 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666694 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666696 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666698 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666701 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666704 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666706 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 01:29:50.761388 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666709 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666711 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666713 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666716 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666718 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666721 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666724 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666726 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666729 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666731 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666734 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666737 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666739 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666742 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666744 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666748 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666752 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666754 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666757 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666760 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 01:29:50.762267 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666762 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666765 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666767 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666771 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666774 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666777 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666780 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666782 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666786 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666788 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666791 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666794 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666796 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666799 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666801 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.666804 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667172 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667177 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667181 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 01:29:50.763150 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667183 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667186 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667189 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667192 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667194 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667196 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667199 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667202 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667204 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667207 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667210 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667214 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667217 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667220 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667222 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667225 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667228 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667231 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667233 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667235 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 01:29:50.764208 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667238 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667242 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667245 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667249 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667252 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667254 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667256 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667259 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667262 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667264 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667268 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667270 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667273 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667275 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667278 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667280 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667282 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667285 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667287 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 01:29:50.765074 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667290 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667292 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667294 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667297 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667299 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667302 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667304 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667306 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667309 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667311 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667314 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667316 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667318 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667321 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667326 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667328 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667332 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667334 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667336 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667339 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 01:29:50.765937 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667341 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667343 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667346 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667348 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667351 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667353 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667356 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667358 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667360 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667363 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667365 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667367 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667370 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667374 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667378 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667381 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667384 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667386 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667388 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 01:29:50.766811 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667391 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667393 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667395 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667398 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667400 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667403 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667405 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667409 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667411 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667415 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667417 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667420 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667422 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667425 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667427 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667430 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667432 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.667435 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667517 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667527 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 01:29:50.767660 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667536 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667543 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667547 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667550 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667554 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667557 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667560 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667563 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667566 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667569 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667572 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667575 2564 flags.go:64] FLAG: --cgroup-root="" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667578 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667581 2564 flags.go:64] FLAG: --client-ca-file="" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667584 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667586 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667590 2564 flags.go:64] FLAG: --cluster-domain="" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667592 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667595 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667599 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667603 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667607 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667610 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 01:29:50.768536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667613 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667616 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667619 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667622 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667624 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667627 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667630 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667634 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667637 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667640 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667644 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667647 2564 flags.go:64] FLAG: --enable-server="true" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667650 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667654 2564 flags.go:64] FLAG: --event-burst="100" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667657 2564 flags.go:64] FLAG: --event-qps="50" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667660 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667662 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667666 2564 flags.go:64] FLAG: --eviction-hard="" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667669 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667672 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667674 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667677 2564 flags.go:64] FLAG: --eviction-soft="" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667680 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667682 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667685 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 01:29:50.769641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667688 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667690 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667693 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667697 2564 flags.go:64] FLAG: --feature-gates="" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667700 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667705 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667708 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667711 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667714 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667717 2564 flags.go:64] FLAG: --help="false" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667719 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-138-219.ec2.internal" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667722 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667725 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667727 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667731 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667734 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667737 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667739 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667745 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667748 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667751 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667754 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667757 2564 flags.go:64] FLAG: --kube-reserved="" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667760 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 01:29:50.772850 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667762 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667765 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667768 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667771 2564 flags.go:64] FLAG: --lock-file="" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667773 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667776 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667779 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667783 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667787 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667789 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667792 2564 flags.go:64] FLAG: --logging-format="text" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667796 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667799 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667803 2564 flags.go:64] FLAG: --manifest-url="" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667805 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667809 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667812 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667816 2564 flags.go:64] FLAG: --max-pods="110" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667819 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667822 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667824 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667827 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667830 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667833 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667836 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 01:29:50.774156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667842 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667845 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667848 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667851 2564 flags.go:64] FLAG: --pod-cidr="" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667854 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667859 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667862 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667865 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667868 2564 flags.go:64] FLAG: --port="10250" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667871 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667873 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ccebe7c324bf3eda" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667876 2564 flags.go:64] FLAG: --qos-reserved="" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667879 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667882 2564 flags.go:64] FLAG: --register-node="true" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667884 2564 flags.go:64] FLAG: --register-with-taints="" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667888 2564 flags.go:64] FLAG: --registry-burst="10" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667891 2564 flags.go:64] FLAG: --registry-qps="5" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667894 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667896 2564 flags.go:64] FLAG: --reserved-memory="" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667901 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667906 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667909 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667912 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667915 2564 flags.go:64] FLAG: --runonce="false" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667917 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 01:29:50.781333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667920 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667923 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667926 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667928 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667931 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667934 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667937 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667940 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667942 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667945 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667948 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667968 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667971 2564 flags.go:64] FLAG: --system-cgroups="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667974 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667980 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667982 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667985 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667988 2564 flags.go:64] FLAG: --tls-min-version="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667991 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667994 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667996 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.667999 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.668002 2564 flags.go:64] FLAG: --v="2" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.668005 2564 flags.go:64] FLAG: --version="false" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.668009 2564 flags.go:64] FLAG: --vmodule="" Jul 24 01:29:50.782431 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.668012 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.668017 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668144 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668149 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668152 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668155 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668157 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668160 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668164 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668167 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668170 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668172 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668175 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668178 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668181 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668184 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668186 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668189 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668192 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668195 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 01:29:50.783518 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668197 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668200 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668202 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668204 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668207 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668209 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668211 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668214 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668216 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668219 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668221 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668224 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668226 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668230 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668233 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668236 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668238 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668241 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668243 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 01:29:50.784385 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668246 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668248 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668250 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668253 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668255 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668258 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668260 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668264 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668267 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668270 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668272 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668275 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668278 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668280 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668283 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668285 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668288 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668290 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668293 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 01:29:50.785255 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668295 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668298 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668300 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668303 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668305 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668307 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668310 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668314 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668317 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668320 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668322 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668325 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668328 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668330 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668333 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668337 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668340 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668342 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668345 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668348 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 01:29:50.786142 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668351 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668353 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668356 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668358 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668361 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668363 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668366 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668369 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668371 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668374 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668376 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668379 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668381 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668383 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668386 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668388 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668391 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668393 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668395 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668399 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 01:29:50.787236 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668403 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668405 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.668408 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.668870 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.675497 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.675511 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.675529 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.675535 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675595 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675600 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675604 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675606 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675609 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675612 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 01:29:50.788108 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675614 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675617 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675621 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675624 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675627 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675630 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675633 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675636 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675639 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675641 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675644 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675646 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675649 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675651 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675654 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675656 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675659 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675661 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675663 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 01:29:50.788746 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675666 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675671 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675675 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675683 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675693 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675696 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675700 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675703 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675706 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675709 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675712 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675714 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675718 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675721 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675724 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675726 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675729 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675731 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675734 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 01:29:50.789606 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675739 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675742 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675744 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675747 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675749 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675752 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675754 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675756 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675759 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675761 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675764 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675766 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675769 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675771 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675774 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675779 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675781 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675784 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675791 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675794 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 01:29:50.790467 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675797 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675799 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675801 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675804 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675806 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675809 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675811 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675814 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675816 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675818 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675821 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675823 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675827 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675829 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675831 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675834 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675836 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675839 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675841 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675843 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 01:29:50.791332 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675846 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675848 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675851 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675853 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675855 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675858 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675860 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675864 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675880 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675883 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675886 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675898 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675901 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675904 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.675907 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.675912 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 01:29:50.792201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676090 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676096 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676099 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676101 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676104 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676107 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676109 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676112 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676115 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676118 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676121 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676123 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676125 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676129 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676133 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676136 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676138 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676141 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676143 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 01:29:50.792868 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676145 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676148 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676150 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676152 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676155 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676160 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676162 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676165 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676167 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676175 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676178 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676180 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676183 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676185 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676188 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676190 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676192 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676195 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676197 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676200 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 01:29:50.793729 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676202 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676210 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676212 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676215 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676217 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676220 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676222 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676225 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676227 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676230 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676232 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676235 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676237 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676239 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676242 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676244 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676246 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676250 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676252 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676255 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 01:29:50.794603 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676257 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676260 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676272 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676275 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676278 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676280 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676284 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676287 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676290 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676293 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676295 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676298 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676300 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676304 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676307 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676310 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676312 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676316 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676319 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 01:29:50.933877 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676322 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676324 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676327 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676330 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676332 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676334 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676337 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676339 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676341 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676344 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676347 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676350 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676352 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676354 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676357 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676359 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676367 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676370 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676372 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676374 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 01:29:50.958051 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:50.676377 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.676381 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.676508 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.680790 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.681865 2564 server.go:1019] "Starting client certificate rotation" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.681969 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.682435 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.683718 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.684830 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.690909 2564 log.go:25] "Validated CRI v1 runtime API" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.693105 2564 log.go:25] "Validated CRI v1 image API" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.694836 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 01:29:50.958699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.696714 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cbaa80fe-f716-4f98-a804-0ac3984005f3:/dev/nvme0n1p3 dbcd87ee-00d7-433d-945f-1226d8816eba:/dev/nvme0n1p4] Jul 24 01:29:50.959288 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.696726 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.701896 2564 manager.go:216] Machine: {Timestamp:2026-07-24 01:29:50.700808907 +0000 UTC m=+0.235114406 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098900 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec215ed5bc2ef11ff56c2b976da11ab3 SystemUUID:ec215ed5-bc2e-f11f-f56c-2b976da11ab3 BootID:400dc3b2-ba63-4ddd-a1e5-d02731660a58 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fc:d9:09:65:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fc:d9:09:65:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:e3:f4:47:d2:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.702011 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.702104 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.704533 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.704553 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-219.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.705509 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.705519 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 01:29:50.962207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.705527 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.706011 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.706667 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.709195 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.709706 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.709876 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.709928 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.709992 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.710012 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.710033 2564 kubelet.go:398] "Adding apiserver pod source" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.710044 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.710846 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.710870 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.712896 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.714856 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.714890 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716213 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716229 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716239 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716246 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716257 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716265 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716275 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716283 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716292 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716299 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716311 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.716318 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.717324 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.717334 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.720924 2564 server.go:1268] "Started kubelet" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.720933 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-219.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.721142 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.721187 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.721423 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.721427 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-219.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.721521 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.722546 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.723208 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 24 01:29:50.962544 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.726151 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.726872 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-219.ec2.internal.18c5159cd8ef24f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-219.ec2.internal,UID:ip-10-0-138-219.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-219.ec2.internal,},FirstTimestamp:2026-07-24 01:29:50.72090239 +0000 UTC m=+0.255207890,LastTimestamp:2026-07-24 01:29:50.72090239 +0000 UTC m=+0.255207890,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-219.ec2.internal,}" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.729374 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.729688 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.730075 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.730546 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxv9m" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.730615 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.730621 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.730634 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.730772 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.730785 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731061 2564 factory.go:55] Registering systemd factory Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731074 2564 factory.go:223] Registration of the systemd container factory successfully Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731328 2564 factory.go:153] Registering CRI-O factory Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731341 2564 factory.go:223] Registration of the crio container factory successfully Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731399 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731417 2564 factory.go:103] Registering Raw factory Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731428 2564 manager.go:1196] Started watching for new ooms in manager Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.731868 2564 manager.go:349] Starting recovery of all containers Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.734804 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.735216 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-219.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.741031 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gxv9m" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.743929 2564 manager.go:354] Recovery completed Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.749154 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.750895 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.750925 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.750939 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.751800 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.751811 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.751829 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.751814 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.752672 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.176854 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.752699 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.752712 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.752930 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-219.ec2.internal.18c5159cdab904fe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-219.ec2.internal,UID:ip-10-0-138-219.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-219.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-219.ec2.internal,},FirstTimestamp:2026-07-24 01:29:50.750909694 +0000 UTC m=+0.285215194,LastTimestamp:2026-07-24 01:29:50.750909694 +0000 UTC m=+0.285215194,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-219.ec2.internal,}" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.754460 2564 policy_none.go:49] "None policy: Start" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.754477 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.754490 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.756587 2564 policy_none.go:47] "Start" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.820076 2564 manager.go:337] "Starting Device Plugin manager" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.820119 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.820132 2564 server.go:87] "Starting device plugin registration server" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.820375 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.820386 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.822504 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.822570 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.822578 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.822832 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.822879 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.915600 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.916940 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.916993 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.917022 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.917064 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.918788 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.920543 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.921309 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.921338 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.921349 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.921370 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:50.930313 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.930334 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-219.ec2.internal\": node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:50.945108 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.017581 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal"] Jul 24 01:29:52.178546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.017651 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.019344 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.019367 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.019377 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.021788 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.021936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.021976 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.022617 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.022637 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.022646 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.023127 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.023149 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.023164 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.025524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.025554 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.026279 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientMemory" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.026300 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.026329 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeHasSufficientPID" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.032648 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1de7be92a0a914801205d3ece54cca0e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-219.ec2.internal\" (UID: \"1de7be92a0a914801205d3ece54cca0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.032668 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3deac22f55b27978c45b7ec5b962b9ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal\" (UID: \"3deac22f55b27978c45b7ec5b962b9ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:51.032684 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3deac22f55b27978c45b7ec5b962b9ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal\" (UID: \"3deac22f55b27978c45b7ec5b962b9ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:51.045765 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:51.052378 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-219.ec2.internal\" not found" node="ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.179768 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:51.055592 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-219.ec2.internal\" not found" node="ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.202876 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1de7be92a0a914801205d3ece54cca0e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-219.ec2.internal\" (UID: \"1de7be92a0a914801205d3ece54cca0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.202909 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3deac22f55b27978c45b7ec5b962b9ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal\" (UID: \"3deac22f55b27978c45b7ec5b962b9ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.202933 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3deac22f55b27978c45b7ec5b962b9ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal\" (UID: \"3deac22f55b27978c45b7ec5b962b9ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.202976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3deac22f55b27978c45b7ec5b962b9ed-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal\" (UID: \"3deac22f55b27978c45b7ec5b962b9ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.202981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1de7be92a0a914801205d3ece54cca0e-config\") pod \"kube-apiserver-proxy-ip-10-0-138-219.ec2.internal\" (UID: \"1de7be92a0a914801205d3ece54cca0e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.202981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3deac22f55b27978c45b7ec5b962b9ed-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal\" (UID: \"3deac22f55b27978c45b7ec5b962b9ed\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.310622 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.215956 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.415696 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.316058 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.418497 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.416513 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.500801 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.423671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.500801 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.427321 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:52.621177 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.516697 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.621177 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.617177 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.779683 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.717626 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.779683 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.750913 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 01:29:52.779683 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.751053 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 01:29:52.863339 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.818485 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:52.863339 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.822626 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 01:24:51 +0000 UTC" deadline="2028-04-21 00:25:47.132250326 +0000 UTC" Jul 24 01:29:52.863339 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:52.822650 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15286h55m54.309603813s" Jul 24 01:29:52.949469 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:52.919267 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:53.069845 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.020063 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:53.077408 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:53.077172 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3deac22f55b27978c45b7ec5b962b9ed.slice/crio-12e191ac4ad86383566e079cc71d69b5728ff4be8de4dc20a3ae485f4e00cbfe WatchSource:0}: Error finding container 12e191ac4ad86383566e079cc71d69b5728ff4be8de4dc20a3ae485f4e00cbfe: Status 404 returned error can't find the container with id 12e191ac4ad86383566e079cc71d69b5728ff4be8de4dc20a3ae485f4e00cbfe Jul 24 01:29:53.242040 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:53.077563 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de7be92a0a914801205d3ece54cca0e.slice/crio-e3f400b4558f0490217de0feaaf261d6776e729d417c9e9e1aae7f7edf14f409 WatchSource:0}: Error finding container e3f400b4558f0490217de0feaaf261d6776e729d417c9e9e1aae7f7edf14f409: Status 404 returned error can't find the container with id e3f400b4558f0490217de0feaaf261d6776e729d417c9e9e1aae7f7edf14f409 Jul 24 01:29:53.242040 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.081613 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 01:29:53.242040 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.122970 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:53.242040 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.158424 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 01:29:53.242040 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.223869 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:53.336525 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.277145 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 01:29:53.336525 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.324805 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-219.ec2.internal\" not found" Jul 24 01:29:53.496342 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.347435 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 01:29:53.496342 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.399475 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" Jul 24 01:29:53.496342 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.408910 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 01:29:53.496342 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.409655 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" Jul 24 01:29:53.496342 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.417261 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 01:29:53.780888 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.780858 2564 apiserver.go:52] "Watching apiserver" Jul 24 01:29:53.787725 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.787697 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 01:29:53.788095 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.788069 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zx5s5","kube-system/konnectivity-agent-7g7tz","kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal","openshift-multus/multus-additional-cni-plugins-fzw96","openshift-cluster-node-tuning-operator/tuned-l7lll","openshift-dns/node-resolver-jh57j","openshift-image-registry/node-ca-qwnc7","openshift-multus/multus-jxrgw","openshift-multus/network-metrics-daemon-jz7sv","openshift-network-diagnostics/network-check-target-g4q7p","openshift-network-operator/iptables-alerter-h69xc","openshift-ovn-kubernetes/ovnkube-node-bt6k6"] Jul 24 01:29:53.790632 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.790608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.794036 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.793771 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:53.794036 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.793846 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:29:53.797318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.794607 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.797318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.794808 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.797318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.794998 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qxrd4\"" Jul 24 01:29:53.797318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.795370 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.798847 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.797611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.803823 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.803800 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.803987 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.803915 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 01:29:53.804053 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.804022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mhpgb\"" Jul 24 01:29:53.804279 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.803807 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 01:29:53.804936 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.804446 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.806224 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.806205 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.808322 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.808359 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.808632 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.808644 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.808889 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809236 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809469 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809486 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809557 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809636 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-82jpg\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809762 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809846 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809858 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.809997 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.810015 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n2m47\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.810065 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-thcns\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.810187 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.810219 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.810466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.810469 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:53.811522 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811443 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.811601 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811559 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811719 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wgtgz\"" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811752 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-kubelet\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-log-socket\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/54cc477a-48f2-42bd-a455-5c27655cf4d5-serviceca\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811854 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811892 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-var-lib-kubelet\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811967 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-run-netns\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.812096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.811992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-device-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.812029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysctl-conf\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.812644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-run\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.812678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-lib-modules\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.812142 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.812718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-etc-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.812151 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813142 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pxfl\" (UniqueName: \"kubernetes.io/projected/f3172562-d5fd-49f2-b690-e620b7629414-kube-api-access-2pxfl\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813208 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-systemd\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckbw5\" (UniqueName: \"kubernetes.io/projected/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-kube-api-access-ckbw5\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.813273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813275 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813305 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl8dl\" (UniqueName: \"kubernetes.io/projected/51017506-0417-48e7-9d6d-08a40ca55e44-kube-api-access-jl8dl\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813334 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-cni-netd\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813364 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck7sc\" (UniqueName: \"kubernetes.io/projected/54cc477a-48f2-42bd-a455-5c27655cf4d5-kube-api-access-ck7sc\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-registration-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813417 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90c50dab-83ca-4c12-a20b-7a904c62617d-tmp\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813446 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-node-log\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813508 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-cni-bin\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813536 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9m4d\" (UniqueName: \"kubernetes.io/projected/90c50dab-83ca-4c12-a20b-7a904c62617d-kube-api-access-h9m4d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813570 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-ovnkube-script-lib\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813615 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-host\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-systemd\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813665 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wdwv\" (UniqueName: \"kubernetes.io/projected/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-kube-api-access-2wdwv\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813730 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cnibin\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.813814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813792 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysctl-d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-tuned\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-slash\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.813974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-ovn\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.814012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-run-ovn-kubernetes\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.814062 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.814229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t44w7\"" Jul 24 01:29:53.814543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.814243 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.814050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-ovnkube-config\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-sys-fs\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815315 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815338 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-sys\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815359 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-systemd-units\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815393 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-socket-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-hosts-file\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-system-cni-dir\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-modprobe-d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-var-lib-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-os-release\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.816337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bst4w\" (UniqueName: \"kubernetes.io/projected/73544e93-3fd9-4035-bdf1-0e04d8f703d1-kube-api-access-bst4w\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-tmp-dir\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysconfig\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-env-overrides\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54cc477a-48f2-42bd-a455-5c27655cf4d5-host\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-kubernetes\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815762 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.816971 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.815796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3172562-d5fd-49f2-b690-e620b7629414-ovn-node-metrics-cert\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.820039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.817723 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.820039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.818456 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 01:29:53.820039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.818972 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 01:29:53.820039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.819112 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d8b67\"" Jul 24 01:29:53.820039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.819248 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 01:29:53.820513 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.820496 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 01:29:53.821349 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.820971 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cbtqp\"" Jul 24 01:29:53.821349 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.821190 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 01:29:53.821349 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.821238 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:53.821349 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.821300 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:29:53.823699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.823680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:53.823788 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.823739 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:29:53.835784 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.835762 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hk5g7" Jul 24 01:29:53.842858 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.842813 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hk5g7" Jul 24 01:29:53.901653 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.901592 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 01:29:53.916225 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-systemd-units\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916330 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916240 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916330 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-socket-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.916330 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-systemd-units\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916330 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916290 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-hosts-file\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.916330 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-system-cni-dir\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916344 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-cni-multus\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916346 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-hosts-file\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-hostroot\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916345 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-modprobe-d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916374 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-system-cni-dir\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916397 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-var-lib-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916430 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-var-lib-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-socket-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-os-release\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916469 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bst4w\" (UniqueName: \"kubernetes.io/projected/73544e93-3fd9-4035-bdf1-0e04d8f703d1-kube-api-access-bst4w\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916485 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-modprobe-d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.916547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916509 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-os-release\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916561 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d1321dd5-5c89-4ca8-bbc2-c7a398ed9940-konnectivity-ca\") pod \"konnectivity-agent-7g7tz\" (UID: \"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940\") " pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-tmp-dir\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysconfig\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916793 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysconfig\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916865 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-env-overrides\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.916945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54cc477a-48f2-42bd-a455-5c27655cf4d5-host\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5m9h\" (UniqueName: \"kubernetes.io/projected/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-kube-api-access-x5m9h\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.916917 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-tmp-dir\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917005 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917049 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-kubernetes\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917057 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/54cc477a-48f2-42bd-a455-5c27655cf4d5-host\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917076 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.917094 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917179 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3172562-d5fd-49f2-b690-e620b7629414-ovn-node-metrics-cert\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-kubernetes\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-host-slash\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:53.917247 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:53.917258 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:29:54.41723586 +0000 UTC m=+2.882370271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-kubelet\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917286 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917404 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-kubelet\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917433 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-log-socket\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917501 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-kubelet\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917503 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-env-overrides\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/54cc477a-48f2-42bd-a455-5c27655cf4d5-serviceca\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917541 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-log-socket\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917632 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.917772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-var-lib-kubelet\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917768 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-var-lib-kubelet\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917812 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-run-netns\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917840 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-run-netns\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917875 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-device-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysctl-conf\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-device-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/54cc477a-48f2-42bd-a455-5c27655cf4d5-serviceca\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-run\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.917989 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-lib-modules\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-etc-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918044 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysctl-conf\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918058 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pxfl\" (UniqueName: \"kubernetes.io/projected/f3172562-d5fd-49f2-b690-e620b7629414-kube-api-access-2pxfl\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-run\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918101 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-etc-openvswitch\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.918487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d1321dd5-5c89-4ca8-bbc2-c7a398ed9940-agent-certs\") pod \"konnectivity-agent-7g7tz\" (UID: \"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940\") " pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918180 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-cni-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-lib-modules\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-systemd\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckbw5\" (UniqueName: \"kubernetes.io/projected/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-kube-api-access-ckbw5\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-systemd\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-iptables-alerter-script\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918393 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918423 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-cnibin\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918458 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-socket-dir-parent\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-dbus\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918492 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl8dl\" (UniqueName: \"kubernetes.io/projected/51017506-0417-48e7-9d6d-08a40ca55e44-kube-api-access-jl8dl\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:53.919291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-cni-netd\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918687 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-cni-netd\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck7sc\" (UniqueName: \"kubernetes.io/projected/54cc477a-48f2-42bd-a455-5c27655cf4d5-kube-api-access-ck7sc\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918730 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-registration-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-system-cni-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-k8s-cni-cncf-io\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918803 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-cni-bin\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90c50dab-83ca-4c12-a20b-7a904c62617d-tmp\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918833 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-registration-dir\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-node-log\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-cni-bin\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-cni-bin\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918922 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-node-log\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.918954 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-cni-binary-copy\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.919990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-multus-certs\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-kubelet-config\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9m4d\" (UniqueName: \"kubernetes.io/projected/90c50dab-83ca-4c12-a20b-7a904c62617d-kube-api-access-h9m4d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-ovnkube-script-lib\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919310 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-netns\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-host\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919437 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-systemd\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wdwv\" (UniqueName: \"kubernetes.io/projected/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-kube-api-access-2wdwv\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919480 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-host\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cnibin\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919529 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-systemd\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-daemon-config\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-etc-kubernetes\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.920700 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysctl-d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-tuned\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919642 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cnibin\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919645 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-slash\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-slash\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-conf-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-ovnkube-script-lib\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-sysctl-d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-ovn\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-run-ovn-kubernetes\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-run-ovn\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-ovnkube-config\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.919974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3172562-d5fd-49f2-b690-e620b7629414-host-run-ovn-kubernetes\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-sys-fs\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.921392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920136 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-sys-fs\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmp5g\" (UniqueName: \"kubernetes.io/projected/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-kube-api-access-fmp5g\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-os-release\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-sys\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920432 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90c50dab-83ca-4c12-a20b-7a904c62617d-sys\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920541 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3172562-d5fd-49f2-b690-e620b7629414-ovnkube-config\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.922064 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.920637 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/73544e93-3fd9-4035-bdf1-0e04d8f703d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.923632 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.923608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90c50dab-83ca-4c12-a20b-7a904c62617d-etc-tuned\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.925886 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.925447 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90c50dab-83ca-4c12-a20b-7a904c62617d-tmp\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.925886 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.925735 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3172562-d5fd-49f2-b690-e620b7629414-ovn-node-metrics-cert\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.936061 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.930821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bst4w\" (UniqueName: \"kubernetes.io/projected/73544e93-3fd9-4035-bdf1-0e04d8f703d1-kube-api-access-bst4w\") pod \"multus-additional-cni-plugins-fzw96\" (UID: \"73544e93-3fd9-4035-bdf1-0e04d8f703d1\") " pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:53.941508 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.939361 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9m4d\" (UniqueName: \"kubernetes.io/projected/90c50dab-83ca-4c12-a20b-7a904c62617d-kube-api-access-h9m4d\") pod \"tuned-l7lll\" (UID: \"90c50dab-83ca-4c12-a20b-7a904c62617d\") " pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:53.941508 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.940013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl8dl\" (UniqueName: \"kubernetes.io/projected/51017506-0417-48e7-9d6d-08a40ca55e44-kube-api-access-jl8dl\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:53.941508 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.940324 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck7sc\" (UniqueName: \"kubernetes.io/projected/54cc477a-48f2-42bd-a455-5c27655cf4d5-kube-api-access-ck7sc\") pod \"node-ca-qwnc7\" (UID: \"54cc477a-48f2-42bd-a455-5c27655cf4d5\") " pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:53.941508 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.940431 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wdwv\" (UniqueName: \"kubernetes.io/projected/1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d-kube-api-access-2wdwv\") pod \"node-resolver-jh57j\" (UID: \"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d\") " pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:53.941508 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.940474 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pxfl\" (UniqueName: \"kubernetes.io/projected/f3172562-d5fd-49f2-b690-e620b7629414-kube-api-access-2pxfl\") pod \"ovnkube-node-bt6k6\" (UID: \"f3172562-d5fd-49f2-b690-e620b7629414\") " pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:53.941508 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.941002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckbw5\" (UniqueName: \"kubernetes.io/projected/fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0-kube-api-access-ckbw5\") pod \"aws-ebs-csi-driver-node-drl2p\" (UID: \"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:53.991676 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.991624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" event={"ID":"1de7be92a0a914801205d3ece54cca0e","Type":"ContainerStarted","Data":"e3f400b4558f0490217de0feaaf261d6776e729d417c9e9e1aae7f7edf14f409"} Jul 24 01:29:53.992860 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:53.992836 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" event={"ID":"3deac22f55b27978c45b7ec5b962b9ed","Type":"ContainerStarted","Data":"12e191ac4ad86383566e079cc71d69b5728ff4be8de4dc20a3ae485f4e00cbfe"} Jul 24 01:29:54.021520 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d1321dd5-5c89-4ca8-bbc2-c7a398ed9940-agent-certs\") pod \"konnectivity-agent-7g7tz\" (UID: \"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940\") " pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:54.021520 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-cni-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-iptables-alerter-script\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021570 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-cnibin\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-socket-dir-parent\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-dbus\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021622 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-system-cni-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021643 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-k8s-cni-cncf-io\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-cni-bin\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-cni-binary-copy\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021709 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-multus-certs\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021732 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-kubelet-config\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:54.021751 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-netns\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-daemon-config\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-etc-kubernetes\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021851 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-cnibin\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-conf-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021893 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-conf-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmp5g\" (UniqueName: \"kubernetes.io/projected/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-kube-api-access-fmp5g\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-os-release\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-cni-multus\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021979 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-system-cni-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-hostroot\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d1321dd5-5c89-4ca8-bbc2-c7a398ed9940-konnectivity-ca\") pod \"konnectivity-agent-7g7tz\" (UID: \"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940\") " pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-multus-certs\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022098 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5m9h\" (UniqueName: \"kubernetes.io/projected/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-kube-api-access-x5m9h\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:54.022284 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.022179 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-host-slash\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022213 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-kubelet\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.022246 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:29:54.522229982 +0000 UTC m=+2.987364379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022368 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-dbus\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022424 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-netns\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022425 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-iptables-alerter-script\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022487 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-socket-dir-parent\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022507 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-etc-kubernetes\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022804 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-host-slash\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-kubelet\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021937 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-run-k8s-cni-cncf-io\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.021907 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-cni-dir\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-os-release\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-multus-daemon-config\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023718 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-cni-binary-copy\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023718 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022990 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-kubelet-config\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:54.023718 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022022 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-cni-bin\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023718 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.022994 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-hostroot\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023718 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.023026 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-host-var-lib-cni-multus\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.023718 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.023168 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d1321dd5-5c89-4ca8-bbc2-c7a398ed9940-konnectivity-ca\") pod \"konnectivity-agent-7g7tz\" (UID: \"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940\") " pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:54.025037 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.025003 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d1321dd5-5c89-4ca8-bbc2-c7a398ed9940-agent-certs\") pod \"konnectivity-agent-7g7tz\" (UID: \"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940\") " pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:54.031086 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.031032 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:29:54.031086 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.031058 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:29:54.031086 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.031072 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:54.031293 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.031192 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:29:54.531172366 +0000 UTC m=+2.996306778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:54.033264 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.033239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmp5g\" (UniqueName: \"kubernetes.io/projected/ed43dff0-8f5a-481e-be8e-13bc60ed8c2c-kube-api-access-fmp5g\") pod \"iptables-alerter-h69xc\" (UID: \"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c\") " pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.033871 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.033852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5m9h\" (UniqueName: \"kubernetes.io/projected/ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7-kube-api-access-x5m9h\") pod \"multus-jxrgw\" (UID: \"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7\") " pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.084529 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.084494 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 01:29:54.113969 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.113889 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l7lll" Jul 24 01:29:54.125987 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.125960 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:29:54.135826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.135805 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" Jul 24 01:29:54.150171 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.150143 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jh57j" Jul 24 01:29:54.157793 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.157776 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fzw96" Jul 24 01:29:54.165341 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.165322 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwnc7" Jul 24 01:29:54.173915 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.173890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h69xc" Jul 24 01:29:54.184478 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.184461 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:29:54.190137 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.190101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jxrgw" Jul 24 01:29:54.425072 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.425045 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:54.425239 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.425206 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:54.425277 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.425264 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:29:55.425247144 +0000 UTC m=+3.890381555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:54.525637 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.525599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:54.525778 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.525719 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:54.525875 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.525783 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:29:55.525763223 +0000 UTC m=+3.990897622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:54.626345 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.626310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:54.626532 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.626510 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:29:54.626581 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.626540 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:29:54.626581 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.626554 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:54.626659 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.626618 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:29:55.626598279 +0000 UTC m=+4.091732675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:54.845813 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.845729 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 01:24:53 +0000 UTC" deadline="2028-03-06 08:01:46.633035478 +0000 UTC" Jul 24 01:29:54.845813 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.845761 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14190h31m51.787279036s" Jul 24 01:29:54.986367 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:54.986338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:54.986776 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:54.986437 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:29:55.019017 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:55.018979 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3172562_d5fd_49f2_b690_e620b7629414.slice/crio-774e24027b70428082307afb9a4875aeb26cb2d8bf3e9dd4a8bca99005add3fb WatchSource:0}: Error finding container 774e24027b70428082307afb9a4875aeb26cb2d8bf3e9dd4a8bca99005add3fb: Status 404 returned error can't find the container with id 774e24027b70428082307afb9a4875aeb26cb2d8bf3e9dd4a8bca99005add3fb Jul 24 01:29:55.021281 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:55.021250 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54cc477a_48f2_42bd_a455_5c27655cf4d5.slice/crio-25dbf2223293145f7bf12b9fc4d64794f9131f1b7de2e042a3df521aa7a98759 WatchSource:0}: Error finding container 25dbf2223293145f7bf12b9fc4d64794f9131f1b7de2e042a3df521aa7a98759: Status 404 returned error can't find the container with id 25dbf2223293145f7bf12b9fc4d64794f9131f1b7de2e042a3df521aa7a98759 Jul 24 01:29:55.023221 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:55.023017 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6f4c5f_9eaa_42d8_a59d_3b70fc77e1f0.slice/crio-3a412fc256ed0a9fa737c302cdec6bc07b4aa9713393cd1aa679110e30d421a9 WatchSource:0}: Error finding container 3a412fc256ed0a9fa737c302cdec6bc07b4aa9713393cd1aa679110e30d421a9: Status 404 returned error can't find the container with id 3a412fc256ed0a9fa737c302cdec6bc07b4aa9713393cd1aa679110e30d421a9 Jul 24 01:29:55.025604 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:55.025583 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc4afbc_9f30_49fb_a31f_2ef91e6f5e8d.slice/crio-2fea3c793cd937d1f5a34093307149150caaea598355cdcd5acde0d07d4ec168 WatchSource:0}: Error finding container 2fea3c793cd937d1f5a34093307149150caaea598355cdcd5acde0d07d4ec168: Status 404 returned error can't find the container with id 2fea3c793cd937d1f5a34093307149150caaea598355cdcd5acde0d07d4ec168 Jul 24 01:29:55.026971 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:55.026948 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73544e93_3fd9_4035_bdf1_0e04d8f703d1.slice/crio-16d015f972bc7fa5444c9292d40fc2978f317ad393fae2f393704899a7d74318 WatchSource:0}: Error finding container 16d015f972bc7fa5444c9292d40fc2978f317ad393fae2f393704899a7d74318: Status 404 returned error can't find the container with id 16d015f972bc7fa5444c9292d40fc2978f317ad393fae2f393704899a7d74318 Jul 24 01:29:55.028353 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:29:55.028097 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded43dff0_8f5a_481e_be8e_13bc60ed8c2c.slice/crio-d7d5b3e571f515f812f14d4956a333786ae4bcb65ce6de878b39c1a08b77fa23 WatchSource:0}: Error finding container d7d5b3e571f515f812f14d4956a333786ae4bcb65ce6de878b39c1a08b77fa23: Status 404 returned error can't find the container with id d7d5b3e571f515f812f14d4956a333786ae4bcb65ce6de878b39c1a08b77fa23 Jul 24 01:29:55.432380 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.432113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:55.432557 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.432329 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:55.432557 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.432504 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:29:57.432481724 +0000 UTC m=+5.897616131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:55.533526 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.533483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:55.533710 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.533647 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:55.533710 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.533709 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:29:57.533689782 +0000 UTC m=+5.998824182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:55.634754 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.634621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:55.634918 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.634793 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:29:55.634918 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.634813 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:29:55.634918 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.634825 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:55.634918 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.634879 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:29:57.634861864 +0000 UTC m=+6.099996272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:55.846015 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.845906 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 01:24:53 +0000 UTC" deadline="2028-01-13 11:11:12.984921964 +0000 UTC" Jul 24 01:29:55.846015 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.845972 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12921h41m17.138955061s" Jul 24 01:29:55.988332 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.988303 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:55.988892 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.988437 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:29:55.989037 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:55.989017 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:55.989167 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:55.989146 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:29:56.009489 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.009411 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7g7tz" event={"ID":"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940","Type":"ContainerStarted","Data":"f4ad61eaa4bf0a3f51f876a330e94a375b8522ff3bc991192ac11320bac74d93"} Jul 24 01:29:56.016071 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.015992 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jxrgw" event={"ID":"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7","Type":"ContainerStarted","Data":"c71212d297cdfac8b21d8658fada31daf0eb5b36a789552867d15e30afe30f7e"} Jul 24 01:29:56.033407 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.033348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h69xc" event={"ID":"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c","Type":"ContainerStarted","Data":"d7d5b3e571f515f812f14d4956a333786ae4bcb65ce6de878b39c1a08b77fa23"} Jul 24 01:29:56.042865 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.042821 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwnc7" event={"ID":"54cc477a-48f2-42bd-a455-5c27655cf4d5","Type":"ContainerStarted","Data":"25dbf2223293145f7bf12b9fc4d64794f9131f1b7de2e042a3df521aa7a98759"} Jul 24 01:29:56.054379 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.054349 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" event={"ID":"1de7be92a0a914801205d3ece54cca0e","Type":"ContainerStarted","Data":"e0b50b70c048f40c50231cc436b977e15765efd0cc8bffe6978751ee20647234"} Jul 24 01:29:56.066369 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.066341 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jh57j" event={"ID":"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d","Type":"ContainerStarted","Data":"2fea3c793cd937d1f5a34093307149150caaea598355cdcd5acde0d07d4ec168"} Jul 24 01:29:56.079812 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.079789 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" event={"ID":"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0","Type":"ContainerStarted","Data":"3a412fc256ed0a9fa737c302cdec6bc07b4aa9713393cd1aa679110e30d421a9"} Jul 24 01:29:56.090042 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.090015 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"774e24027b70428082307afb9a4875aeb26cb2d8bf3e9dd4a8bca99005add3fb"} Jul 24 01:29:56.098350 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.096547 2564 generic.go:356] "Generic (PLEG): container finished" podID="3deac22f55b27978c45b7ec5b962b9ed" containerID="c4d4cb4a8eaec70102604fdb16b757a3e4c4d9bdd042d9c6c3cd4548f639263b" exitCode=0 Jul 24 01:29:56.098350 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.096622 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" event={"ID":"3deac22f55b27978c45b7ec5b962b9ed","Type":"ContainerDied","Data":"c4d4cb4a8eaec70102604fdb16b757a3e4c4d9bdd042d9c6c3cd4548f639263b"} Jul 24 01:29:56.105113 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.104421 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l7lll" event={"ID":"90c50dab-83ca-4c12-a20b-7a904c62617d","Type":"ContainerStarted","Data":"ff61b9309997282fe4a4845b5a22364b5cf51a0ce0a68c9b090017f45350cbc2"} Jul 24 01:29:56.107922 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.107900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerStarted","Data":"16d015f972bc7fa5444c9292d40fc2978f317ad393fae2f393704899a7d74318"} Jul 24 01:29:56.112450 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.112402 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-219.ec2.internal" podStartSLOduration=3.112385033 podStartE2EDuration="3.112385033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:29:56.069155809 +0000 UTC m=+4.534290228" watchObservedRunningTime="2026-07-24 01:29:56.112385033 +0000 UTC m=+4.577519453" Jul 24 01:29:56.986476 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:56.986437 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:56.986647 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:56.986564 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:29:57.116321 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.116232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" event={"ID":"3deac22f55b27978c45b7ec5b962b9ed","Type":"ContainerStarted","Data":"59d5484e31004d64d3e429a5d66d61269a9a50be8c275f433e062daf8f6fb357"} Jul 24 01:29:57.131710 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.131418 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-219.ec2.internal" podStartSLOduration=4.131398569 podStartE2EDuration="4.131398569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:29:57.130180597 +0000 UTC m=+5.595315016" watchObservedRunningTime="2026-07-24 01:29:57.131398569 +0000 UTC m=+5.596532987" Jul 24 01:29:57.450300 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.449672 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:57.450300 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.449835 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:57.450300 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.449905 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:01.449886044 +0000 UTC m=+9.915020640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:29:57.550917 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.550400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:57.550917 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.550528 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:57.550917 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.550598 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:30:01.550578173 +0000 UTC m=+10.015712583 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:29:57.651626 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.651587 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:57.651803 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.651748 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:29:57.651803 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.651768 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:29:57.651803 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.651780 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:57.651962 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.651840 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:30:01.651821253 +0000 UTC m=+10.116955674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:29:57.989091 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.989061 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:57.989271 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.989217 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:29:57.989337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:57.989297 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:57.989429 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:57.989384 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:29:58.987241 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:58.987199 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:29:58.987726 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:58.987340 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:29:59.987745 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:59.987259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:29:59.987745 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:59.987396 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:29:59.987745 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:29:59.987745 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:29:59.988405 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:29:59.987847 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:00.987253 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:00.986990 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:00.987253 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:00.987140 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:01.485543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:01.485501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:01.486025 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.485642 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:30:01.486025 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.485702 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:09.485687308 +0000 UTC m=+17.950821705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:30:01.586512 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:01.586470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:01.586706 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.586681 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:30:01.586784 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.586770 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:30:09.58674911 +0000 UTC m=+18.051883506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:30:01.687450 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:01.687412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:01.687617 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.687568 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:30:01.687617 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.687582 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:30:01.687617 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.687590 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:30:01.687786 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.687634 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:30:09.687621697 +0000 UTC m=+18.152756091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:30:01.988013 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:01.987983 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:01.988192 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.988103 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:01.988192 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:01.988174 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:01.988277 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:01.988263 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:02.987079 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:02.986593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:02.987079 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:02.986737 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:03.990269 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:03.990236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:03.990742 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:03.990362 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:03.990742 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:03.990726 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:03.990861 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:03.990823 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:04.986883 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:04.986847 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:04.987033 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:04.986964 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:05.989923 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:05.989890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:05.990346 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:05.989890 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:05.990346 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:05.989996 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:05.990346 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:05.990140 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:06.986616 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:06.986576 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:06.986781 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:06.986705 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:07.986936 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:07.986896 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:07.987409 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:07.986903 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:07.987409 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:07.987038 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:07.987409 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:07.987154 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:08.987119 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:08.987084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:08.987505 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:08.987200 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:09.545993 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:09.545967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:09.546120 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.546106 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:30:09.546187 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.546177 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:25.546159725 +0000 UTC m=+34.011294131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:30:09.646649 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:09.646618 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:09.646792 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.646766 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:30:09.646852 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.646831 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:30:25.646813383 +0000 UTC m=+34.111947803 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:30:09.747400 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:09.747367 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:09.747595 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.747517 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:30:09.747595 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.747543 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:30:09.747595 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.747556 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:30:09.747739 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.747605 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:30:25.747591285 +0000 UTC m=+34.212725681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:30:09.987028 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:09.986983 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:09.987238 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.987140 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:09.987238 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:09.987175 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:09.987611 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:09.987317 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:10.986664 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:10.986627 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:10.986850 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:10.986763 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:11.987677 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:11.987644 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:11.988150 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:11.987817 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:11.988150 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:11.987872 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:11.988150 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:11.987981 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:12.987156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:12.987110 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:12.987315 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:12.987251 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:13.986618 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:13.986453 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:13.987253 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:13.986532 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:13.987253 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:13.986693 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:13.987253 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:13.986773 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:14.151308 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.151283 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" event={"ID":"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0","Type":"ContainerStarted","Data":"76ec0770c8f76ce06fc34815960d0643e0e45843b6a7ea273f0ea9d41e7acd61"} Jul 24 01:30:14.152910 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.152894 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:30:14.153259 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.153242 2564 generic.go:356] "Generic (PLEG): container finished" podID="f3172562-d5fd-49f2-b690-e620b7629414" containerID="6d8c1b1696ac974f46d5caa6a8021933a314f27b325813ad6f4a2070b697d2ec" exitCode=1 Jul 24 01:30:14.153328 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.153310 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"4edac28fea7706d337391cd40952c4d39fbb7d95da485a99d5a0117ac4328ba2"} Jul 24 01:30:14.153393 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.153380 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerDied","Data":"6d8c1b1696ac974f46d5caa6a8021933a314f27b325813ad6f4a2070b697d2ec"} Jul 24 01:30:14.153441 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.153429 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"342eae5b48e6a22571a49027b3bc957e2b3749058f4a0262e72b9b7e60fcb1fb"} Jul 24 01:30:14.154491 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.154472 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l7lll" event={"ID":"90c50dab-83ca-4c12-a20b-7a904c62617d","Type":"ContainerStarted","Data":"c5a8e30927490eee14a83d74556daa0b3e122fbd8a56862d60b30450c67966fd"} Jul 24 01:30:14.155787 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.155770 2564 generic.go:356] "Generic (PLEG): container finished" podID="73544e93-3fd9-4035-bdf1-0e04d8f703d1" containerID="9728a5d80b0ef511f9eef5a74a47a04aec90056cd56e47edc5cdbc162dd71787" exitCode=0 Jul 24 01:30:14.155856 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.155844 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerDied","Data":"9728a5d80b0ef511f9eef5a74a47a04aec90056cd56e47edc5cdbc162dd71787"} Jul 24 01:30:14.157298 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.157273 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7g7tz" event={"ID":"d1321dd5-5c89-4ca8-bbc2-c7a398ed9940","Type":"ContainerStarted","Data":"14b45e8b64a7a75971bbe23869ea261ca188a2e109afa9b22b55ddbc616e0c3d"} Jul 24 01:30:14.158953 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.158824 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jxrgw" event={"ID":"ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7","Type":"ContainerStarted","Data":"48c1ba77d6d05ddb9a14182ab092dad7aaab120a56d8c60ef82acfaf6afda459"} Jul 24 01:30:14.160382 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.160357 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwnc7" event={"ID":"54cc477a-48f2-42bd-a455-5c27655cf4d5","Type":"ContainerStarted","Data":"a1fe0ad003daed1f1278847f7c72e5f524e5f3fd379fb1e6764a1a1a948cc914"} Jul 24 01:30:14.161678 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.161658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jh57j" event={"ID":"1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d","Type":"ContainerStarted","Data":"dadaa1f79520756491163fcad40359086548a8aea7dbcbe71ff58558b5c939d2"} Jul 24 01:30:14.189040 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.188997 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qwnc7" podStartSLOduration=3.682427692 podStartE2EDuration="22.188983248s" totalImagesPullingTime="18.506555556s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:14.188732937 +0000 UTC m=+22.653867354" watchObservedRunningTime="2026-07-24 01:30:14.188983248 +0000 UTC m=+22.654117665" Jul 24 01:30:14.203269 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.203225 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7g7tz" podStartSLOduration=7.682200316 podStartE2EDuration="22.203207614s" totalImagesPullingTime="14.521007298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:14.202981589 +0000 UTC m=+22.668116007" watchObservedRunningTime="2026-07-24 01:30:14.203207614 +0000 UTC m=+22.668342035" Jul 24 01:30:14.221053 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.220998 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l7lll" podStartSLOduration=3.726500633 podStartE2EDuration="22.22097879s" totalImagesPullingTime="18.494478157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:14.220886825 +0000 UTC m=+22.686021253" watchObservedRunningTime="2026-07-24 01:30:14.22097879 +0000 UTC m=+22.686113202" Jul 24 01:30:14.240598 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.240551 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jxrgw" podStartSLOduration=3.722528809 podStartE2EDuration="22.240532715s" totalImagesPullingTime="18.518003906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:14.240239746 +0000 UTC m=+22.705374164" watchObservedRunningTime="2026-07-24 01:30:14.240532715 +0000 UTC m=+22.705667132" Jul 24 01:30:14.254764 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.254724 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jh57j" podStartSLOduration=3.7523656990000003 podStartE2EDuration="22.254708534s" totalImagesPullingTime="18.502342835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:14.254411334 +0000 UTC m=+22.719545769" watchObservedRunningTime="2026-07-24 01:30:14.254708534 +0000 UTC m=+22.719842951" Jul 24 01:30:14.986416 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:14.986390 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:14.986540 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:14.986522 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:15.002945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.002823 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 01:30:15.166163 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.166115 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" event={"ID":"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0","Type":"ContainerStarted","Data":"f4cb8598095c786252296e8281211c68e10136238aa4a61e3188628bb426c43f"} Jul 24 01:30:15.169270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.169248 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:30:15.169710 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.169677 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"27b25d005d95b0d15c02a2d52461aa4d535f968cbc5bd69a64581a6b7b96fff9"} Jul 24 01:30:15.169818 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.169765 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"a971d3a4a80bda2da16178bbcf9f023ac47c6ee18574c265a5f94fdb37ac64a4"} Jul 24 01:30:15.169880 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.169816 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"49ab7c438c136f91794472226ce5efb55cb4857a4ca8b268b5cd4c81b00ad4e6"} Jul 24 01:30:15.171404 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.171359 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h69xc" event={"ID":"ed43dff0-8f5a-481e-be8e-13bc60ed8c2c","Type":"ContainerStarted","Data":"c94796635f367432d412cc4a86926d5a037de6a26036c482f64c939144f60610"} Jul 24 01:30:15.186480 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.186440 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h69xc" podStartSLOduration=4.681172675 podStartE2EDuration="23.186427453s" totalImagesPullingTime="18.505254778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:15.18629862 +0000 UTC m=+23.651433038" watchObservedRunningTime="2026-07-24 01:30:15.186427453 +0000 UTC m=+23.651561870" Jul 24 01:30:15.903290 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.903185 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T01:30:15.002840707Z","UUID":"6782b319-aed1-4662-8acf-bef0740b8ca1","Handler":null,"Name":"","Endpoint":""} Jul 24 01:30:15.904866 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.904845 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 01:30:15.904992 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.904885 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 01:30:15.987350 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.987322 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:15.987554 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:15.987454 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:15.987554 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:15.987514 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:15.987667 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:15.987623 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:16.175814 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:16.175752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" event={"ID":"fe6f4c5f-9eaa-42d8-a59d-3b70fc77e1f0","Type":"ContainerStarted","Data":"8787fa3653bcd6e64c61faccb22fae6570b879d22f1b64d39ab03b52a052ed91"} Jul 24 01:30:16.192557 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:16.192517 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-drl2p" podStartSLOduration=3.478970124 podStartE2EDuration="24.192501718s" totalImagesPullingTime="20.713531594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:16.19188249 +0000 UTC m=+24.657016908" watchObservedRunningTime="2026-07-24 01:30:16.192501718 +0000 UTC m=+24.657636158" Jul 24 01:30:16.764365 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:16.764146 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:30:16.764704 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:16.764683 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:30:16.986634 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:16.986603 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:16.986814 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:16.986706 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:17.182842 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:17.182811 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:30:17.183265 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:17.183155 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"536b4601d4a21612e96c36a07cf484b79d807d1abf0656bcabc8b22479f453f3"} Jul 24 01:30:17.183556 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:17.183518 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:30:17.184002 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:17.183984 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7g7tz" Jul 24 01:30:17.987433 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:17.987397 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:17.987638 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:17.987546 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:17.987638 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:17.987572 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:17.987757 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:17.987704 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:18.987025 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:18.986771 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:18.987814 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:18.987045 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:19.192793 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.192761 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:30:19.193192 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.193160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"0da80e86a67cdd691a10e01252166f72ee8af56e7b95a39cd8d4148253a5dee3"} Jul 24 01:30:19.193425 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.193403 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:30:19.193730 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.193707 2564 scope.go:117] "RemoveContainer" containerID="6d8c1b1696ac974f46d5caa6a8021933a314f27b325813ad6f4a2070b697d2ec" Jul 24 01:30:19.195083 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.195062 2564 generic.go:356] "Generic (PLEG): container finished" podID="73544e93-3fd9-4035-bdf1-0e04d8f703d1" containerID="54992b8b62956fe74cad0e04b7fd1d04a633a3e51a52027c4f9c348e10ecb82a" exitCode=0 Jul 24 01:30:19.195185 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.195158 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerDied","Data":"54992b8b62956fe74cad0e04b7fd1d04a633a3e51a52027c4f9c348e10ecb82a"} Jul 24 01:30:19.209642 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.209624 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:30:19.987592 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.987527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:19.988076 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:19.987655 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:19.988076 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:19.987782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:19.988076 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:19.987924 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:20.201000 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.200972 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:30:20.201396 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.201360 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" event={"ID":"f3172562-d5fd-49f2-b690-e620b7629414","Type":"ContainerStarted","Data":"f0a560b7aab87bc7b00f8779be48a959d9db4a05df4636cc1585c1a314bf2b87"} Jul 24 01:30:20.201634 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.201613 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:30:20.201726 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.201679 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:30:20.217163 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.217140 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:30:20.228115 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.228072 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" podStartSLOduration=9.433101829 podStartE2EDuration="28.228058766s" totalImagesPullingTime="18.794956937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:20.226450782 +0000 UTC m=+28.691585254" watchObservedRunningTime="2026-07-24 01:30:20.228058766 +0000 UTC m=+28.693193182" Jul 24 01:30:20.743969 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.743940 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zx5s5"] Jul 24 01:30:20.744140 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.744057 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:20.744244 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:20.744215 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:20.747404 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.747376 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g4q7p"] Jul 24 01:30:20.747514 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.747496 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:20.747599 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:20.747579 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:20.748271 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.748250 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jz7sv"] Jul 24 01:30:20.748363 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:20.748341 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:20.748453 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:20.748439 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:21.208002 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:21.207969 2564 generic.go:356] "Generic (PLEG): container finished" podID="73544e93-3fd9-4035-bdf1-0e04d8f703d1" containerID="be5b8cde82f5b50d44de20cc2515af67f4c5807bd16552ed02b4c33f297e8f14" exitCode=0 Jul 24 01:30:21.208554 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:21.208046 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerDied","Data":"be5b8cde82f5b50d44de20cc2515af67f4c5807bd16552ed02b4c33f297e8f14"} Jul 24 01:30:21.988143 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:21.988110 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:21.988325 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:21.988217 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:21.988325 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:21.988306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:21.988441 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:21.988402 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:22.986900 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:22.986863 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:22.987399 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:22.986968 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:23.214970 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:23.214932 2564 generic.go:356] "Generic (PLEG): container finished" podID="73544e93-3fd9-4035-bdf1-0e04d8f703d1" containerID="fe9627b36def43ffbb0f3d27ef51d633fe0f0a9789cfed4a4c82c0202f857889" exitCode=0 Jul 24 01:30:23.215137 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:23.214990 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerDied","Data":"fe9627b36def43ffbb0f3d27ef51d633fe0f0a9789cfed4a4c82c0202f857889"} Jul 24 01:30:23.987194 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:23.987156 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:23.987194 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:23.987188 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:23.987969 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:23.987279 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:23.987969 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:23.987460 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:24.987371 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:24.987338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:24.987752 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:24.987450 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g4q7p" podUID="a5fd18a6-cd32-4522-ba55-37d366a7ff8b" Jul 24 01:30:25.571857 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:25.571817 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:25.572041 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.571990 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:30:25.572097 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.572053 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs podName:51017506-0417-48e7-9d6d-08a40ca55e44 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:57.572038077 +0000 UTC m=+66.037172475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs") pod "network-metrics-daemon-jz7sv" (UID: "51017506-0417-48e7-9d6d-08a40ca55e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 01:30:25.672423 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:25.672386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:25.672588 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.672545 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 01:30:25.672655 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.672619 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret podName:f843c8af-66c4-41df-bc00-7f2f53f5d3cd nodeName:}" failed. No retries permitted until 2026-07-24 01:30:57.672599841 +0000 UTC m=+66.137734261 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret") pod "global-pull-secret-syncer-zx5s5" (UID: "f843c8af-66c4-41df-bc00-7f2f53f5d3cd") : object "kube-system"/"original-pull-secret" not registered Jul 24 01:30:25.773563 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:25.773526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:25.773750 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.773727 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 01:30:25.773803 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.773761 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 01:30:25.773803 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.773776 2564 projected.go:196] Error preparing data for projected volume kube-api-access-tx6s5 for pod openshift-network-diagnostics/network-check-target-g4q7p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:30:25.773897 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.773834 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5 podName:a5fd18a6-cd32-4522-ba55-37d366a7ff8b nodeName:}" failed. No retries permitted until 2026-07-24 01:30:57.773820022 +0000 UTC m=+66.238954417 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tx6s5" (UniqueName: "kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5") pod "network-check-target-g4q7p" (UID: "a5fd18a6-cd32-4522-ba55-37d366a7ff8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 01:30:25.987447 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:25.987419 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:25.987801 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.987527 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jz7sv" podUID="51017506-0417-48e7-9d6d-08a40ca55e44" Jul 24 01:30:25.987801 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:25.987589 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:25.987801 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:25.987691 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zx5s5" podUID="f843c8af-66c4-41df-bc00-7f2f53f5d3cd" Jul 24 01:30:26.390277 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.390241 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-219.ec2.internal" event="NodeReady" Jul 24 01:30:26.390456 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.390402 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 01:30:26.435216 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.435180 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9mtdc"] Jul 24 01:30:26.438139 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.438093 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hl8p7"] Jul 24 01:30:26.438270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.438222 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.440311 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.440292 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.441040 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.440804 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 01:30:26.441040 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.440855 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g8g76\"" Jul 24 01:30:26.441040 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.440855 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 01:30:26.442835 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.442814 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 01:30:26.442835 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.442828 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tfscd\"" Jul 24 01:30:26.442987 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.442940 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 01:30:26.443048 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.442999 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 01:30:26.445579 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.445559 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hl8p7"] Jul 24 01:30:26.453730 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.448953 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9mtdc"] Jul 24 01:30:26.579988 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.579766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85471a1f-0293-4586-8794-1f26399f854a-cert\") pod \"ingress-canary-hl8p7\" (UID: \"85471a1f-0293-4586-8794-1f26399f854a\") " pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.580191 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.580008 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a505e1-5606-461f-814c-b2af6874eaf7-config-volume\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.580191 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.580047 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2a505e1-5606-461f-814c-b2af6874eaf7-tmp-dir\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.580191 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.580066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.580191 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.580091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8h9s\" (UniqueName: \"kubernetes.io/projected/a2a505e1-5606-461f-814c-b2af6874eaf7-kube-api-access-p8h9s\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.580406 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.580198 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2trb\" (UniqueName: \"kubernetes.io/projected/85471a1f-0293-4586-8794-1f26399f854a-kube-api-access-f2trb\") pod \"ingress-canary-hl8p7\" (UID: \"85471a1f-0293-4586-8794-1f26399f854a\") " pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.681090 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681000 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2trb\" (UniqueName: \"kubernetes.io/projected/85471a1f-0293-4586-8794-1f26399f854a-kube-api-access-f2trb\") pod \"ingress-canary-hl8p7\" (UID: \"85471a1f-0293-4586-8794-1f26399f854a\") " pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.681090 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85471a1f-0293-4586-8794-1f26399f854a-cert\") pod \"ingress-canary-hl8p7\" (UID: \"85471a1f-0293-4586-8794-1f26399f854a\") " pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.681090 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681087 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a505e1-5606-461f-814c-b2af6874eaf7-config-volume\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.681401 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2a505e1-5606-461f-814c-b2af6874eaf7-tmp-dir\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.681401 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.681401 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:26.681249 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 01:30:26.681401 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:26.681308 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls podName:a2a505e1-5606-461f-814c-b2af6874eaf7 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:27.181289825 +0000 UTC m=+35.646424240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls") pod "dns-default-9mtdc" (UID: "a2a505e1-5606-461f-814c-b2af6874eaf7") : secret "dns-default-metrics-tls" not found Jul 24 01:30:26.681401 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8h9s\" (UniqueName: \"kubernetes.io/projected/a2a505e1-5606-461f-814c-b2af6874eaf7-kube-api-access-p8h9s\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.681607 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681585 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2a505e1-5606-461f-814c-b2af6874eaf7-tmp-dir\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.681798 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.681772 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2a505e1-5606-461f-814c-b2af6874eaf7-config-volume\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.686117 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.686094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85471a1f-0293-4586-8794-1f26399f854a-cert\") pod \"ingress-canary-hl8p7\" (UID: \"85471a1f-0293-4586-8794-1f26399f854a\") " pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.689974 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.689949 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8h9s\" (UniqueName: \"kubernetes.io/projected/a2a505e1-5606-461f-814c-b2af6874eaf7-kube-api-access-p8h9s\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:26.690561 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.690535 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2trb\" (UniqueName: \"kubernetes.io/projected/85471a1f-0293-4586-8794-1f26399f854a-kube-api-access-f2trb\") pod \"ingress-canary-hl8p7\" (UID: \"85471a1f-0293-4586-8794-1f26399f854a\") " pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.760732 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.760696 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hl8p7" Jul 24 01:30:26.986655 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.986547 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:26.989769 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.989745 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 01:30:26.990148 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.989746 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 01:30:26.990148 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:26.989746 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c62r2\"" Jul 24 01:30:27.185533 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.185491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:27.185689 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:27.185668 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 01:30:27.185763 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:27.185750 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls podName:a2a505e1-5606-461f-814c-b2af6874eaf7 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:28.185727654 +0000 UTC m=+36.650862051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls") pod "dns-default-9mtdc" (UID: "a2a505e1-5606-461f-814c-b2af6874eaf7") : secret "dns-default-metrics-tls" not found Jul 24 01:30:27.255998 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.255975 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hl8p7"] Jul 24 01:30:27.260020 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:27.259999 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85471a1f_0293_4586_8794_1f26399f854a.slice/crio-9cca16510f5c04e967729e83beb89fc40765558352fa525e228003c26d30d7d6 WatchSource:0}: Error finding container 9cca16510f5c04e967729e83beb89fc40765558352fa525e228003c26d30d7d6: Status 404 returned error can't find the container with id 9cca16510f5c04e967729e83beb89fc40765558352fa525e228003c26d30d7d6 Jul 24 01:30:27.986717 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.986626 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:27.986861 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.986833 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:27.989343 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.989316 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 01:30:27.989343 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.989323 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 01:30:27.990305 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:27.990281 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pnzwr\"" Jul 24 01:30:28.193511 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:28.193473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:28.193697 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:28.193619 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 01:30:28.193697 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:28.193687 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls podName:a2a505e1-5606-461f-814c-b2af6874eaf7 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:30.193666061 +0000 UTC m=+38.658800463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls") pod "dns-default-9mtdc" (UID: "a2a505e1-5606-461f-814c-b2af6874eaf7") : secret "dns-default-metrics-tls" not found Jul 24 01:30:28.230840 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:28.230790 2564 generic.go:356] "Generic (PLEG): container finished" podID="73544e93-3fd9-4035-bdf1-0e04d8f703d1" containerID="47187c7ac3538445ed72a5347c95764c6f55f926be017a611074b46f7fe061be" exitCode=0 Jul 24 01:30:28.231000 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:28.230867 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerDied","Data":"47187c7ac3538445ed72a5347c95764c6f55f926be017a611074b46f7fe061be"} Jul 24 01:30:28.232154 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:28.232074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hl8p7" event={"ID":"85471a1f-0293-4586-8794-1f26399f854a","Type":"ContainerStarted","Data":"9cca16510f5c04e967729e83beb89fc40765558352fa525e228003c26d30d7d6"} Jul 24 01:30:29.018837 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.018802 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd"] Jul 24 01:30:29.023384 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.023358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" Jul 24 01:30:29.025950 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.025924 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 01:30:29.026098 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.026076 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 01:30:29.026208 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.026076 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6nqlz\"" Jul 24 01:30:29.030036 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.029986 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd"] Jul 24 01:30:29.201875 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.201831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzmqb\" (UniqueName: \"kubernetes.io/projected/0fe852ed-61d6-4476-8756-a8ddf0a29110-kube-api-access-tzmqb\") pod \"migrator-6865c9967-ll8vd\" (UID: \"0fe852ed-61d6-4476-8756-a8ddf0a29110\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" Jul 24 01:30:29.238145 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.238090 2564 generic.go:356] "Generic (PLEG): container finished" podID="73544e93-3fd9-4035-bdf1-0e04d8f703d1" containerID="24b9ac885069f89dafa6085a1ccd292be10cd014a7490313f6545f0567e2a154" exitCode=0 Jul 24 01:30:29.238291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.238178 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerDied","Data":"24b9ac885069f89dafa6085a1ccd292be10cd014a7490313f6545f0567e2a154"} Jul 24 01:30:29.302399 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.302299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzmqb\" (UniqueName: \"kubernetes.io/projected/0fe852ed-61d6-4476-8756-a8ddf0a29110-kube-api-access-tzmqb\") pod \"migrator-6865c9967-ll8vd\" (UID: \"0fe852ed-61d6-4476-8756-a8ddf0a29110\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" Jul 24 01:30:29.312713 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.312686 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzmqb\" (UniqueName: \"kubernetes.io/projected/0fe852ed-61d6-4476-8756-a8ddf0a29110-kube-api-access-tzmqb\") pod \"migrator-6865c9967-ll8vd\" (UID: \"0fe852ed-61d6-4476-8756-a8ddf0a29110\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" Jul 24 01:30:29.334641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.334610 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" Jul 24 01:30:29.579624 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.579553 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd"] Jul 24 01:30:29.583883 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:29.583863 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe852ed_61d6_4476_8756_a8ddf0a29110.slice/crio-ec7fe166232726dc99cc3f1bdd726e53049dad1e580f3b47c76f6e077b58f9a5 WatchSource:0}: Error finding container ec7fe166232726dc99cc3f1bdd726e53049dad1e580f3b47c76f6e077b58f9a5: Status 404 returned error can't find the container with id ec7fe166232726dc99cc3f1bdd726e53049dad1e580f3b47c76f6e077b58f9a5 Jul 24 01:30:29.586436 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.586399 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t59rx"] Jul 24 01:30:29.589487 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.589469 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.592141 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.591945 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 01:30:29.592141 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.591945 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 01:30:29.592141 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.591977 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pds6h\"" Jul 24 01:30:29.592141 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.591948 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 01:30:29.592141 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.591995 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 01:30:29.599204 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.599184 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t59rx"] Jul 24 01:30:29.705743 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.705688 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45c169e4-62e2-49f9-bdef-43796a5877f5-data-volume\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.705743 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.705739 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.705743 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.705755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjpht\" (UniqueName: \"kubernetes.io/projected/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-api-access-tjpht\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.705979 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.705874 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.705979 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.705901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45c169e4-62e2-49f9-bdef-43796a5877f5-crio-socket\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.806658 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.806621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45c169e4-62e2-49f9-bdef-43796a5877f5-data-volume\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.806658 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.806659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.806853 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.806679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjpht\" (UniqueName: \"kubernetes.io/projected/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-api-access-tjpht\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.806853 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:29.806766 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 01:30:29.806914 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.806848 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.806914 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:29.806875 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:30.306855192 +0000 UTC m=+38.771989599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : configmap "kube-rbac-proxy" not found Jul 24 01:30:29.806914 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.806908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45c169e4-62e2-49f9-bdef-43796a5877f5-crio-socket\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.807024 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:29.806930 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 01:30:29.807024 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.806943 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45c169e4-62e2-49f9-bdef-43796a5877f5-data-volume\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.807024 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:29.806984 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:30.306964649 +0000 UTC m=+38.772099045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : secret "insights-runtime-extractor-tls" not found Jul 24 01:30:29.807196 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.807178 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45c169e4-62e2-49f9-bdef-43796a5877f5-crio-socket\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:29.816119 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:29.816103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjpht\" (UniqueName: \"kubernetes.io/projected/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-api-access-tjpht\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:30.028514 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.028484 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-swqqh"] Jul 24 01:30:30.045291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.045267 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-swqqh"] Jul 24 01:30:30.045418 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.045361 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.048039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.048015 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 01:30:30.048039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.048032 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 01:30:30.048259 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.048050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 01:30:30.048259 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.048097 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 01:30:30.048259 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.048050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f292s\"" Jul 24 01:30:30.211004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.210968 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-signing-cabundle\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.211004 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.211011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:30.211256 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.211093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llt9w\" (UniqueName: \"kubernetes.io/projected/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-kube-api-access-llt9w\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.211256 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:30.211103 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 01:30:30.211256 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.211175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-signing-key\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.211256 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:30.211184 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls podName:a2a505e1-5606-461f-814c-b2af6874eaf7 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:34.211170227 +0000 UTC m=+42.676304622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls") pod "dns-default-9mtdc" (UID: "a2a505e1-5606-461f-814c-b2af6874eaf7") : secret "dns-default-metrics-tls" not found Jul 24 01:30:30.243268 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.243237 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hl8p7" event={"ID":"85471a1f-0293-4586-8794-1f26399f854a","Type":"ContainerStarted","Data":"e4548f3cf96713033839bd37c24176319a5694f19ca1cd567f765820e684808a"} Jul 24 01:30:30.247300 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.247247 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fzw96" event={"ID":"73544e93-3fd9-4035-bdf1-0e04d8f703d1","Type":"ContainerStarted","Data":"e0b670b457176415cacdbfd6cae92fdc3320c99743315d099a8bf22c6c9521f6"} Jul 24 01:30:30.248421 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.248400 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" event={"ID":"0fe852ed-61d6-4476-8756-a8ddf0a29110","Type":"ContainerStarted","Data":"ec7fe166232726dc99cc3f1bdd726e53049dad1e580f3b47c76f6e077b58f9a5"} Jul 24 01:30:30.259749 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.259700 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hl8p7" podStartSLOduration=2.020028561 podStartE2EDuration="4.259682579s" totalImagesPullingTime="2.239654018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:30.259050207 +0000 UTC m=+38.724184651" watchObservedRunningTime="2026-07-24 01:30:30.259682579 +0000 UTC m=+38.724816998" Jul 24 01:30:30.280179 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.280082 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fzw96" podStartSLOduration=8.861611222 podStartE2EDuration="38.280066868s" totalImagesPullingTime="29.418455646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:30.279703369 +0000 UTC m=+38.744837788" watchObservedRunningTime="2026-07-24 01:30:30.280066868 +0000 UTC m=+38.745201285" Jul 24 01:30:30.312408 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.312375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:30.312570 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.312462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-signing-cabundle\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.312570 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.312506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:30.312570 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.312536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llt9w\" (UniqueName: \"kubernetes.io/projected/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-kube-api-access-llt9w\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.312726 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.312571 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-signing-key\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.312726 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:30.312597 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 01:30:30.312726 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:30.312675 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:31.312652959 +0000 UTC m=+39.777787374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : configmap "kube-rbac-proxy" not found Jul 24 01:30:30.313001 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:30.312974 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 01:30:30.313112 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:30.313062 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:31.313024836 +0000 UTC m=+39.778159248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : secret "insights-runtime-extractor-tls" not found Jul 24 01:30:30.313735 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.313706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-signing-cabundle\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.315630 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.315601 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-signing-key\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.327977 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.327952 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llt9w\" (UniqueName: \"kubernetes.io/projected/1fb9f32a-87d3-4103-ba12-af0ce4b20bcc-kube-api-access-llt9w\") pod \"service-ca-658d7b7c78-swqqh\" (UID: \"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc\") " pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.353908 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.353874 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" Jul 24 01:30:30.508622 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:30.508594 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-swqqh"] Jul 24 01:30:30.513355 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:30.513326 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fb9f32a_87d3_4103_ba12_af0ce4b20bcc.slice/crio-cdbf2d48a8b096bbc6b9cafd8794a585336ed4e96cc69711aaf92ad0d1d26bc1 WatchSource:0}: Error finding container cdbf2d48a8b096bbc6b9cafd8794a585336ed4e96cc69711aaf92ad0d1d26bc1: Status 404 returned error can't find the container with id cdbf2d48a8b096bbc6b9cafd8794a585336ed4e96cc69711aaf92ad0d1d26bc1 Jul 24 01:30:31.252616 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:31.252578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" event={"ID":"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc","Type":"ContainerStarted","Data":"cdbf2d48a8b096bbc6b9cafd8794a585336ed4e96cc69711aaf92ad0d1d26bc1"} Jul 24 01:30:31.321175 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:31.321147 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:31.321307 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:31.321239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:31.321307 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:31.321277 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 01:30:31.321584 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:31.321335 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 01:30:31.321584 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:31.321353 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:33.321331239 +0000 UTC m=+41.786465651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : configmap "kube-rbac-proxy" not found Jul 24 01:30:31.321584 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:31.321373 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:33.32136291 +0000 UTC m=+41.786497310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : secret "insights-runtime-extractor-tls" not found Jul 24 01:30:32.167874 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:32.167706 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jh57j_1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d/dns-node-resolver/0.log" Jul 24 01:30:32.258033 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:32.258004 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" event={"ID":"0fe852ed-61d6-4476-8756-a8ddf0a29110","Type":"ContainerStarted","Data":"0f0d22ac999e0d0d7121559f0eb2601f596ec885bb9d05daa8bb21ee4b4bf42b"} Jul 24 01:30:32.258514 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:32.258056 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" event={"ID":"0fe852ed-61d6-4476-8756-a8ddf0a29110","Type":"ContainerStarted","Data":"651c756605e9fd9797610a283efbd81e102fcc60a29e5894d61588d8ba6b7d74"} Jul 24 01:30:32.275154 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:32.275093 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-ll8vd" podStartSLOduration=1.603407554 podStartE2EDuration="3.27507456s" totalImagesPullingTime="1.671667006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:32.273605614 +0000 UTC m=+40.738740030" watchObservedRunningTime="2026-07-24 01:30:32.27507456 +0000 UTC m=+40.740208978" Jul 24 01:30:33.262812 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:33.262771 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" event={"ID":"1fb9f32a-87d3-4103-ba12-af0ce4b20bcc","Type":"ContainerStarted","Data":"8916df0710dbb47db9b8da262556f04274ac02e8dbe5c284704b8ce8c664c3ba"} Jul 24 01:30:33.281576 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:33.281527 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-swqqh" podStartSLOduration=0.666662235 podStartE2EDuration="3.281511067s" totalImagesPullingTime="2.614848832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:30:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:33.280056994 +0000 UTC m=+41.745191426" watchObservedRunningTime="2026-07-24 01:30:33.281511067 +0000 UTC m=+41.746645462" Jul 24 01:30:33.336424 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:33.336398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:33.336483 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:33.336472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:33.336557 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:33.336533 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 01:30:33.336621 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:33.336606 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 01:30:33.336691 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:33.336625 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:37.336604491 +0000 UTC m=+45.801738888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : configmap "kube-rbac-proxy" not found Jul 24 01:30:33.336691 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:33.336660 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:37.336640109 +0000 UTC m=+45.801774507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : secret "insights-runtime-extractor-tls" not found Jul 24 01:30:33.567596 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:33.567566 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwnc7_54cc477a-48f2-42bd-a455-5c27655cf4d5/node-ca/0.log" Jul 24 01:30:34.167991 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:34.167961 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hl8p7_85471a1f-0293-4586-8794-1f26399f854a/serve-healthcheck-canary/0.log" Jul 24 01:30:34.243411 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:34.243377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:34.243554 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:34.243536 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 01:30:34.243609 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:34.243601 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls podName:a2a505e1-5606-461f-814c-b2af6874eaf7 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:42.243585595 +0000 UTC m=+50.708719989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls") pod "dns-default-9mtdc" (UID: "a2a505e1-5606-461f-814c-b2af6874eaf7") : secret "dns-default-metrics-tls" not found Jul 24 01:30:34.568756 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:34.568683 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-ll8vd_0fe852ed-61d6-4476-8756-a8ddf0a29110/migrator/0.log" Jul 24 01:30:34.767786 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:34.767756 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-ll8vd_0fe852ed-61d6-4476-8756-a8ddf0a29110/graceful-termination/0.log" Jul 24 01:30:37.368494 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:37.368464 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:37.369083 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:37.368534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:37.369083 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:37.368657 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 01:30:37.369083 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:30:37.368712 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls podName:45c169e4-62e2-49f9-bdef-43796a5877f5 nodeName:}" failed. No retries permitted until 2026-07-24 01:30:45.368696297 +0000 UTC m=+53.833830701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t59rx" (UID: "45c169e4-62e2-49f9-bdef-43796a5877f5") : secret "insights-runtime-extractor-tls" not found Jul 24 01:30:37.369264 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:37.369095 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c169e4-62e2-49f9-bdef-43796a5877f5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:42.304364 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:42.304322 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:42.307595 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:42.307568 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2a505e1-5606-461f-814c-b2af6874eaf7-metrics-tls\") pod \"dns-default-9mtdc\" (UID: \"a2a505e1-5606-461f-814c-b2af6874eaf7\") " pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:42.351585 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:42.351552 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:42.478025 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:42.477990 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9mtdc"] Jul 24 01:30:42.481712 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:42.481688 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a505e1_5606_461f_814c_b2af6874eaf7.slice/crio-a3d029f8304d58a8831044bc290a7872f81c5a8d0f9f01bb07de01ec6688ac74 WatchSource:0}: Error finding container a3d029f8304d58a8831044bc290a7872f81c5a8d0f9f01bb07de01ec6688ac74: Status 404 returned error can't find the container with id a3d029f8304d58a8831044bc290a7872f81c5a8d0f9f01bb07de01ec6688ac74 Jul 24 01:30:43.289680 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:43.289640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9mtdc" event={"ID":"a2a505e1-5606-461f-814c-b2af6874eaf7","Type":"ContainerStarted","Data":"a3d029f8304d58a8831044bc290a7872f81c5a8d0f9f01bb07de01ec6688ac74"} Jul 24 01:30:45.296963 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.296872 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9mtdc" event={"ID":"a2a505e1-5606-461f-814c-b2af6874eaf7","Type":"ContainerStarted","Data":"ed0737e8224f0ee2a86f2d22fbc1583c42adc421cbf1b03430f531a5659d08d5"} Jul 24 01:30:45.296963 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.296956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9mtdc" event={"ID":"a2a505e1-5606-461f-814c-b2af6874eaf7","Type":"ContainerStarted","Data":"e21e856c64a5801610b151d38481e15bda169df953643af2650577e29052bf2d"} Jul 24 01:30:45.297434 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.297010 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:45.313616 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.313568 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9mtdc" podStartSLOduration=17.162767375 podStartE2EDuration="19.313555105s" totalImagesPullingTime="2.15078773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:45.31314162 +0000 UTC m=+53.778276032" watchObservedRunningTime="2026-07-24 01:30:45.313555105 +0000 UTC m=+53.778689521" Jul 24 01:30:45.426033 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.425991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:45.428334 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.428308 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c169e4-62e2-49f9-bdef-43796a5877f5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t59rx\" (UID: \"45c169e4-62e2-49f9-bdef-43796a5877f5\") " pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:45.502380 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.502324 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t59rx" Jul 24 01:30:45.621799 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:45.621769 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t59rx"] Jul 24 01:30:45.624528 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:45.624501 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c169e4_62e2_49f9_bdef_43796a5877f5.slice/crio-5b3b7dc3b8f14340e4e195bf7f6349b898eb498ce763be0404c7868fe3bd7290 WatchSource:0}: Error finding container 5b3b7dc3b8f14340e4e195bf7f6349b898eb498ce763be0404c7868fe3bd7290: Status 404 returned error can't find the container with id 5b3b7dc3b8f14340e4e195bf7f6349b898eb498ce763be0404c7868fe3bd7290 Jul 24 01:30:46.301623 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:46.301539 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t59rx" event={"ID":"45c169e4-62e2-49f9-bdef-43796a5877f5","Type":"ContainerStarted","Data":"5b3b7dc3b8f14340e4e195bf7f6349b898eb498ce763be0404c7868fe3bd7290"} Jul 24 01:30:50.454411 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.454379 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x"] Jul 24 01:30:50.458928 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.458899 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j"] Jul 24 01:30:50.459080 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.459042 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.462172 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.462152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.463353 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.463151 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 01:30:50.463353 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.463168 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 24 01:30:50.463353 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.463206 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-m4l6q\"" Jul 24 01:30:50.463353 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.463218 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 01:30:50.463353 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.463261 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 01:30:50.464803 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.464765 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 01:30:50.464803 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.464789 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 01:30:50.464990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.464829 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 01:30:50.464990 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.464873 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 01:30:50.468525 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.468505 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x"] Jul 24 01:30:50.473025 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.473002 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j"] Jul 24 01:30:50.565947 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.565911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.565947 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.565947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4dw8\" (UniqueName: \"kubernetes.io/projected/42b43e7b-9676-448a-8d86-814de5d7d6eb-kube-api-access-j4dw8\") pod \"managed-serviceaccount-addon-agent-85556877b-mt47x\" (UID: \"42b43e7b-9676-448a-8d86-814de5d7d6eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.566333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.565965 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvtdl\" (UniqueName: \"kubernetes.io/projected/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-kube-api-access-lvtdl\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.566333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.566074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/42b43e7b-9676-448a-8d86-814de5d7d6eb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85556877b-mt47x\" (UID: \"42b43e7b-9676-448a-8d86-814de5d7d6eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.566333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.566107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-ca\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.566333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.566146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.566333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.566175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-hub\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.566333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.566192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.666947 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.666924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/42b43e7b-9676-448a-8d86-814de5d7d6eb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85556877b-mt47x\" (UID: \"42b43e7b-9676-448a-8d86-814de5d7d6eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.667063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.666954 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-ca\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.667063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.666975 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.667063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.667000 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-hub\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.667063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.667025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.667307 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.667086 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.667307 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.667113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4dw8\" (UniqueName: \"kubernetes.io/projected/42b43e7b-9676-448a-8d86-814de5d7d6eb-kube-api-access-j4dw8\") pod \"managed-serviceaccount-addon-agent-85556877b-mt47x\" (UID: \"42b43e7b-9676-448a-8d86-814de5d7d6eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.667307 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.667151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvtdl\" (UniqueName: \"kubernetes.io/projected/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-kube-api-access-lvtdl\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.668096 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.668067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.670222 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.670185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-ca\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.670325 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.670263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.670479 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.670462 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/42b43e7b-9676-448a-8d86-814de5d7d6eb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85556877b-mt47x\" (UID: \"42b43e7b-9676-448a-8d86-814de5d7d6eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.670533 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.670478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.670998 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.670970 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-hub\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.675636 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.675615 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4dw8\" (UniqueName: \"kubernetes.io/projected/42b43e7b-9676-448a-8d86-814de5d7d6eb-kube-api-access-j4dw8\") pod \"managed-serviceaccount-addon-agent-85556877b-mt47x\" (UID: \"42b43e7b-9676-448a-8d86-814de5d7d6eb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.675818 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.675796 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvtdl\" (UniqueName: \"kubernetes.io/projected/3eae0329-af6e-4c87-98b0-0c2d47ec46ea-kube-api-access-lvtdl\") pod \"cluster-proxy-proxy-agent-6f457bfcf-js85j\" (UID: \"3eae0329-af6e-4c87-98b0-0c2d47ec46ea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.779661 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.779629 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" Jul 24 01:30:50.785901 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.785874 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" Jul 24 01:30:50.928409 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.928382 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x"] Jul 24 01:30:50.933086 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:50.933052 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42b43e7b_9676_448a_8d86_814de5d7d6eb.slice/crio-3c566e0d84d201e579ce167d6a8a85513528335d09d7972ade2cfe885cd993fb WatchSource:0}: Error finding container 3c566e0d84d201e579ce167d6a8a85513528335d09d7972ade2cfe885cd993fb: Status 404 returned error can't find the container with id 3c566e0d84d201e579ce167d6a8a85513528335d09d7972ade2cfe885cd993fb Jul 24 01:30:50.950389 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:50.950370 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j"] Jul 24 01:30:50.953869 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:50.953841 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eae0329_af6e_4c87_98b0_0c2d47ec46ea.slice/crio-504d62521c4cb07554622ebe778f6d474bfb7767460848c4cb3985edb15d7aa4 WatchSource:0}: Error finding container 504d62521c4cb07554622ebe778f6d474bfb7767460848c4cb3985edb15d7aa4: Status 404 returned error can't find the container with id 504d62521c4cb07554622ebe778f6d474bfb7767460848c4cb3985edb15d7aa4 Jul 24 01:30:51.322067 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:51.322027 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" event={"ID":"42b43e7b-9676-448a-8d86-814de5d7d6eb","Type":"ContainerStarted","Data":"3c566e0d84d201e579ce167d6a8a85513528335d09d7972ade2cfe885cd993fb"} Jul 24 01:30:51.323688 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:51.323661 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t59rx" event={"ID":"45c169e4-62e2-49f9-bdef-43796a5877f5","Type":"ContainerStarted","Data":"1eeb618e694156e7568c791185362b0f1becbe1fe04658906e10cf513b2f1db2"} Jul 24 01:30:51.325006 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:51.324981 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" event={"ID":"3eae0329-af6e-4c87-98b0-0c2d47ec46ea","Type":"ContainerStarted","Data":"504d62521c4cb07554622ebe778f6d474bfb7767460848c4cb3985edb15d7aa4"} Jul 24 01:30:52.235465 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:52.234927 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bt6k6" Jul 24 01:30:55.304228 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:55.304198 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9mtdc" Jul 24 01:30:57.347468 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.347376 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" event={"ID":"42b43e7b-9676-448a-8d86-814de5d7d6eb","Type":"ContainerStarted","Data":"76f48bff879c51f704f8a9b91be0d736fb9f0dbe0a1ba6117d92d738a649ca9b"} Jul 24 01:30:57.349182 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.349155 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t59rx" event={"ID":"45c169e4-62e2-49f9-bdef-43796a5877f5","Type":"ContainerStarted","Data":"aa0e46d53c4c75724aaa12fdfe5ecdaf7dc7f8f21ca909865d30f753ea665d25"} Jul 24 01:30:57.351142 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.351102 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" event={"ID":"3eae0329-af6e-4c87-98b0-0c2d47ec46ea","Type":"ContainerStarted","Data":"7f5913d5d8969add979636a81553651eb3b7b695db65104e15b33dcc72bb1de3"} Jul 24 01:30:57.351218 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.351147 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" event={"ID":"3eae0329-af6e-4c87-98b0-0c2d47ec46ea","Type":"ContainerStarted","Data":"4f114f4ae0a2385df170208ca791e381efff055e98ba0f91a0d8ed5ae7ae60dc"} Jul 24 01:30:57.351218 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.351159 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" event={"ID":"3eae0329-af6e-4c87-98b0-0c2d47ec46ea","Type":"ContainerStarted","Data":"661246e674e9a9a2c87525edada3d7f9d81f06f291db061663c94b9558b92d0d"} Jul 24 01:30:57.362334 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.362293 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85556877b-mt47x" podStartSLOduration=1.234197728 podStartE2EDuration="7.362279547s" totalImagesPullingTime="6.128081819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:57.36149521 +0000 UTC m=+65.826629628" watchObservedRunningTime="2026-07-24 01:30:57.362279547 +0000 UTC m=+65.827413964" Jul 24 01:30:57.378364 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.378320 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f457bfcf-js85j" podStartSLOduration=1.2746130660000001 podStartE2EDuration="7.378302957s" totalImagesPullingTime="6.103689891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:30:57.377677157 +0000 UTC m=+65.842811574" watchObservedRunningTime="2026-07-24 01:30:57.378302957 +0000 UTC m=+65.843437374" Jul 24 01:30:57.625844 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.625755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:57.628378 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.628353 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 01:30:57.638539 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.638511 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51017506-0417-48e7-9d6d-08a40ca55e44-metrics-certs\") pod \"network-metrics-daemon-jz7sv\" (UID: \"51017506-0417-48e7-9d6d-08a40ca55e44\") " pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:57.700346 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.700317 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pnzwr\"" Jul 24 01:30:57.707907 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.707879 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jz7sv" Jul 24 01:30:57.727094 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.727069 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:57.730186 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.730166 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 01:30:57.740139 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.740104 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f843c8af-66c4-41df-bc00-7f2f53f5d3cd-original-pull-secret\") pod \"global-pull-secret-syncer-zx5s5\" (UID: \"f843c8af-66c4-41df-bc00-7f2f53f5d3cd\") " pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:57.828110 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.828070 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:57.831002 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.830977 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 01:30:57.841368 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.841342 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 01:30:57.845098 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.845066 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jz7sv"] Jul 24 01:30:57.848339 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:57.848308 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51017506_0417_48e7_9d6d_08a40ca55e44.slice/crio-33523405abbf3ebdcefedd687778376f517e7cfcc553d97997f612523026b99f WatchSource:0}: Error finding container 33523405abbf3ebdcefedd687778376f517e7cfcc553d97997f612523026b99f: Status 404 returned error can't find the container with id 33523405abbf3ebdcefedd687778376f517e7cfcc553d97997f612523026b99f Jul 24 01:30:57.851877 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.851853 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6s5\" (UniqueName: \"kubernetes.io/projected/a5fd18a6-cd32-4522-ba55-37d366a7ff8b-kube-api-access-tx6s5\") pod \"network-check-target-g4q7p\" (UID: \"a5fd18a6-cd32-4522-ba55-37d366a7ff8b\") " pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:57.899685 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.899655 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c62r2\"" Jul 24 01:30:57.907878 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:57.907851 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:30:58.004906 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:58.004880 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zx5s5" Jul 24 01:30:58.075655 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:58.075449 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g4q7p"] Jul 24 01:30:58.078618 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:58.078591 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fd18a6_cd32_4522_ba55_37d366a7ff8b.slice/crio-67c62ba9789db44a2f9c653713705230da6805ab7a989d56509154f9d6b17689 WatchSource:0}: Error finding container 67c62ba9789db44a2f9c653713705230da6805ab7a989d56509154f9d6b17689: Status 404 returned error can't find the container with id 67c62ba9789db44a2f9c653713705230da6805ab7a989d56509154f9d6b17689 Jul 24 01:30:58.167494 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:58.167415 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zx5s5"] Jul 24 01:30:58.170797 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:30:58.170769 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf843c8af_66c4_41df_bc00_7f2f53f5d3cd.slice/crio-bda69960a4f3e08151ff08f3c5ecd2bca3236d7be9bd9ea772602800e884e1a5 WatchSource:0}: Error finding container bda69960a4f3e08151ff08f3c5ecd2bca3236d7be9bd9ea772602800e884e1a5: Status 404 returned error can't find the container with id bda69960a4f3e08151ff08f3c5ecd2bca3236d7be9bd9ea772602800e884e1a5 Jul 24 01:30:58.356755 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:58.356682 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zx5s5" event={"ID":"f843c8af-66c4-41df-bc00-7f2f53f5d3cd","Type":"ContainerStarted","Data":"bda69960a4f3e08151ff08f3c5ecd2bca3236d7be9bd9ea772602800e884e1a5"} Jul 24 01:30:58.358369 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:58.358340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jz7sv" event={"ID":"51017506-0417-48e7-9d6d-08a40ca55e44","Type":"ContainerStarted","Data":"33523405abbf3ebdcefedd687778376f517e7cfcc553d97997f612523026b99f"} Jul 24 01:30:58.360392 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:30:58.360362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g4q7p" event={"ID":"a5fd18a6-cd32-4522-ba55-37d366a7ff8b","Type":"ContainerStarted","Data":"67c62ba9789db44a2f9c653713705230da6805ab7a989d56509154f9d6b17689"} Jul 24 01:31:04.384654 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.384615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g4q7p" event={"ID":"a5fd18a6-cd32-4522-ba55-37d366a7ff8b","Type":"ContainerStarted","Data":"b3960f2d7e4f14176a5b64a0b5cc44e0d1bb8688fb742d097a6ffce8373964dd"} Jul 24 01:31:04.385168 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.384720 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:31:04.386565 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.386540 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t59rx" event={"ID":"45c169e4-62e2-49f9-bdef-43796a5877f5","Type":"ContainerStarted","Data":"a81688ea3c204a75baf266e16af16f2603e90386aabf24e0b75cdcfcaca68997"} Jul 24 01:31:04.387975 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.387940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zx5s5" event={"ID":"f843c8af-66c4-41df-bc00-7f2f53f5d3cd","Type":"ContainerStarted","Data":"aa6c35641721525952bcbddabd1dbe95139f37e3248d97ff26be4a392c1b0652"} Jul 24 01:31:04.389560 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.389539 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jz7sv" event={"ID":"51017506-0417-48e7-9d6d-08a40ca55e44","Type":"ContainerStarted","Data":"1b6e7ac5db9b3da5c5e8de79afd0b1bd580db9e1e399a07408a4adff8ba587e1"} Jul 24 01:31:04.389651 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.389578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jz7sv" event={"ID":"51017506-0417-48e7-9d6d-08a40ca55e44","Type":"ContainerStarted","Data":"bad88712947fb4b40cfbddf2d9b0055caef262d0b7ff6211a7f373ad16013c5a"} Jul 24 01:31:04.400491 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.400454 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g4q7p" podStartSLOduration=66.66106791 podStartE2EDuration="1m12.400440943s" totalImagesPullingTime="5.739373033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:31:04.399164634 +0000 UTC m=+72.864299068" watchObservedRunningTime="2026-07-24 01:31:04.400440943 +0000 UTC m=+72.865575360" Jul 24 01:31:04.415340 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.415269 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t59rx" podStartSLOduration=17.402262746 podStartE2EDuration="35.415252635s" totalImagesPullingTime="18.012989889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:30:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:31:04.415099145 +0000 UTC m=+72.880233561" watchObservedRunningTime="2026-07-24 01:31:04.415252635 +0000 UTC m=+72.880387053" Jul 24 01:31:04.429389 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.429349 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zx5s5" podStartSLOduration=66.774709913 podStartE2EDuration="1m12.42933684s" totalImagesPullingTime="5.654626927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:31:04.429111636 +0000 UTC m=+72.894246109" watchObservedRunningTime="2026-07-24 01:31:04.42933684 +0000 UTC m=+72.894471250" Jul 24 01:31:04.444284 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:04.444249 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jz7sv" podStartSLOduration=66.487600922 podStartE2EDuration="1m12.444238438s" totalImagesPullingTime="5.956637516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:31:04.443619089 +0000 UTC m=+72.908753505" watchObservedRunningTime="2026-07-24 01:31:04.444238438 +0000 UTC m=+72.909372855" Jul 24 01:31:20.175639 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.175466 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jkk89"] Jul 24 01:31:20.182103 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.182082 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.184965 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.184942 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 01:31:20.185072 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.185038 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 01:31:20.185188 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.185169 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p46tp\"" Jul 24 01:31:20.185260 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.185242 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 01:31:20.185333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.185316 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 01:31:20.185409 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.184950 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 01:31:20.186165 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.186149 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 01:31:20.289119 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-wtmp\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289278 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289278 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289278 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289251 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-root\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289441 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-textfile\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289441 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289344 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-sys\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289441 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289371 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rh57\" (UniqueName: \"kubernetes.io/projected/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-kube-api-access-7rh57\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289548 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.289548 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.289482 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-metrics-client-ca\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390589 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390761 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390598 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-metrics-client-ca\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390761 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390628 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-wtmp\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390762 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-wtmp\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390767 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390822 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.390867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390850 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-root\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390888 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-textfile\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-sys\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.390943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rh57\" (UniqueName: \"kubernetes.io/projected/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-kube-api-access-7rh57\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391063 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.391006 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-root\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391277 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:31:20.391109 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 01:31:20.391277 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:31:20.391198 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls podName:4a3163a2-a71a-44c0-abc5-8f439bc00a3d nodeName:}" failed. No retries permitted until 2026-07-24 01:31:20.891177622 +0000 UTC m=+89.356312021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls") pod "node-exporter-jkk89" (UID: "4a3163a2-a71a-44c0-abc5-8f439bc00a3d") : secret "node-exporter-tls" not found Jul 24 01:31:20.391277 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.391242 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-accelerators-collector-config\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391449 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.391317 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-sys\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391528 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.391509 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-textfile\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.391768 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.391749 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-metrics-client-ca\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.393673 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.393628 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.403119 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.403097 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rh57\" (UniqueName: \"kubernetes.io/projected/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-kube-api-access-7rh57\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.895216 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:20.895182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:20.895399 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:31:20.895322 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 01:31:20.895441 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:31:20.895423 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls podName:4a3163a2-a71a-44c0-abc5-8f439bc00a3d nodeName:}" failed. No retries permitted until 2026-07-24 01:31:21.895406931 +0000 UTC m=+90.360541342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls") pod "node-exporter-jkk89" (UID: "4a3163a2-a71a-44c0-abc5-8f439bc00a3d") : secret "node-exporter-tls" not found Jul 24 01:31:21.904270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:21.904221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:21.906535 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:21.906515 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4a3163a2-a71a-44c0-abc5-8f439bc00a3d-node-exporter-tls\") pod \"node-exporter-jkk89\" (UID: \"4a3163a2-a71a-44c0-abc5-8f439bc00a3d\") " pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:21.992377 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:21.992345 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jkk89" Jul 24 01:31:22.006037 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:31:22.006003 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3163a2_a71a_44c0_abc5_8f439bc00a3d.slice/crio-90ad5f1892f74864fa65fea687715527808c1af212569dbe9583b564dc13f3f0 WatchSource:0}: Error finding container 90ad5f1892f74864fa65fea687715527808c1af212569dbe9583b564dc13f3f0: Status 404 returned error can't find the container with id 90ad5f1892f74864fa65fea687715527808c1af212569dbe9583b564dc13f3f0 Jul 24 01:31:22.447442 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:22.447406 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk89" event={"ID":"4a3163a2-a71a-44c0-abc5-8f439bc00a3d","Type":"ContainerStarted","Data":"90ad5f1892f74864fa65fea687715527808c1af212569dbe9583b564dc13f3f0"} Jul 24 01:31:24.455546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:24.455512 2564 generic.go:356] "Generic (PLEG): container finished" podID="4a3163a2-a71a-44c0-abc5-8f439bc00a3d" containerID="1b83e6519a779627275808d64b51b486086191764cafa759e33dc7fe0441d4da" exitCode=0 Jul 24 01:31:24.455935 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:24.455575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk89" event={"ID":"4a3163a2-a71a-44c0-abc5-8f439bc00a3d","Type":"ContainerDied","Data":"1b83e6519a779627275808d64b51b486086191764cafa759e33dc7fe0441d4da"} Jul 24 01:31:25.460685 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:25.460640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk89" event={"ID":"4a3163a2-a71a-44c0-abc5-8f439bc00a3d","Type":"ContainerStarted","Data":"5f549771ad2095ce2b0625e24e55e6bf6343239b9891b42668ca21c419a97b06"} Jul 24 01:31:25.461067 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:25.460699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jkk89" event={"ID":"4a3163a2-a71a-44c0-abc5-8f439bc00a3d","Type":"ContainerStarted","Data":"c1470522b8bf97931385035e98ba99d113aab3c35517d911df927908f15003ea"} Jul 24 01:31:25.480195 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:25.480114 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jkk89" podStartSLOduration=3.970844702 podStartE2EDuration="5.480099676s" totalImagesPullingTime="1.437583396s" totalInitContainerRuntime="71.671578ms" isStatefulPod=true podCreationTimestamp="2026-07-24 01:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:31:25.478735311 +0000 UTC m=+93.943869751" watchObservedRunningTime="2026-07-24 01:31:25.480099676 +0000 UTC m=+93.945234141" Jul 24 01:31:35.395344 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:31:35.395313 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g4q7p" Jul 24 01:34:44.839793 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.839759 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f"] Jul 24 01:34:44.842657 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.842642 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:44.849208 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.845479 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 01:34:44.849762 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.849738 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-k7xvs\"" Jul 24 01:34:44.849856 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.849738 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 01:34:44.853360 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.853341 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f"] Jul 24 01:34:44.957921 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.957872 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz2wr\" (UniqueName: \"kubernetes.io/projected/04acfcca-b610-4b9f-928f-f9b27a6d8ab7-kube-api-access-zz2wr\") pod \"openshift-lws-operator-fd8ccff4c-9rw6f\" (UID: \"04acfcca-b610-4b9f-928f-f9b27a6d8ab7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:44.958084 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:44.957955 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04acfcca-b610-4b9f-928f-f9b27a6d8ab7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9rw6f\" (UID: \"04acfcca-b610-4b9f-928f-f9b27a6d8ab7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:45.058367 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:45.058327 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz2wr\" (UniqueName: \"kubernetes.io/projected/04acfcca-b610-4b9f-928f-f9b27a6d8ab7-kube-api-access-zz2wr\") pod \"openshift-lws-operator-fd8ccff4c-9rw6f\" (UID: \"04acfcca-b610-4b9f-928f-f9b27a6d8ab7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:45.058535 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:45.058387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04acfcca-b610-4b9f-928f-f9b27a6d8ab7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9rw6f\" (UID: \"04acfcca-b610-4b9f-928f-f9b27a6d8ab7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:45.058731 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:45.058715 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04acfcca-b610-4b9f-928f-f9b27a6d8ab7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9rw6f\" (UID: \"04acfcca-b610-4b9f-928f-f9b27a6d8ab7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:45.066583 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:45.066552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz2wr\" (UniqueName: \"kubernetes.io/projected/04acfcca-b610-4b9f-928f-f9b27a6d8ab7-kube-api-access-zz2wr\") pod \"openshift-lws-operator-fd8ccff4c-9rw6f\" (UID: \"04acfcca-b610-4b9f-928f-f9b27a6d8ab7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:45.156068 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:45.156036 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" Jul 24 01:34:45.275916 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:45.275885 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f"] Jul 24 01:34:45.279201 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:34:45.279177 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04acfcca_b610_4b9f_928f_f9b27a6d8ab7.slice/crio-dac08d8617e72d6268cfcc885f5bed92f23416bb4f1d9a38bac150879479e123 WatchSource:0}: Error finding container dac08d8617e72d6268cfcc885f5bed92f23416bb4f1d9a38bac150879479e123: Status 404 returned error can't find the container with id dac08d8617e72d6268cfcc885f5bed92f23416bb4f1d9a38bac150879479e123 Jul 24 01:34:46.087167 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:46.087110 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" event={"ID":"04acfcca-b610-4b9f-928f-f9b27a6d8ab7","Type":"ContainerStarted","Data":"dac08d8617e72d6268cfcc885f5bed92f23416bb4f1d9a38bac150879479e123"} Jul 24 01:34:48.096214 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:48.096175 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" event={"ID":"04acfcca-b610-4b9f-928f-f9b27a6d8ab7","Type":"ContainerStarted","Data":"8d0cb646d2311b07cb92def4dcf59966a07f28640cc45d66421d5f494abdd0c8"} Jul 24 01:34:48.110770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:48.110723 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9rw6f" podStartSLOduration=1.415493994 podStartE2EDuration="4.110708729s" totalImagesPullingTime="2.695214735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:34:48.110324873 +0000 UTC m=+296.575459305" watchObservedRunningTime="2026-07-24 01:34:48.110708729 +0000 UTC m=+296.575843145" Jul 24 01:34:51.813032 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:51.813006 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:34:51.815654 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:51.815633 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 01:34:51.817398 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:34:51.817372 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:35:08.989965 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.989931 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9"] Jul 24 01:35:08.993285 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.993268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:08.996111 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.996090 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ztd2k\"" Jul 24 01:35:08.996254 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.996141 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 01:35:08.996254 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.996162 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 01:35:08.996254 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.996231 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 01:35:08.996409 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:08.996263 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 01:35:09.017399 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.017374 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9"] Jul 24 01:35:09.126657 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.126614 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktfqp\" (UniqueName: \"kubernetes.io/projected/8af867d2-6806-4022-ba32-6941d91bfb99-kube-api-access-ktfqp\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.126867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.126749 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8af867d2-6806-4022-ba32-6941d91bfb99-tmp\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.126867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.126790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8af867d2-6806-4022-ba32-6941d91bfb99-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.126867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.126848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8af867d2-6806-4022-ba32-6941d91bfb99-manifests\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.127007 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.126877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8af867d2-6806-4022-ba32-6941d91bfb99-webhook-cert\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.227721 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.227694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8af867d2-6806-4022-ba32-6941d91bfb99-webhook-cert\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.227897 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.227735 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktfqp\" (UniqueName: \"kubernetes.io/projected/8af867d2-6806-4022-ba32-6941d91bfb99-kube-api-access-ktfqp\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.227897 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.227778 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8af867d2-6806-4022-ba32-6941d91bfb99-tmp\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.227897 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.227796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8af867d2-6806-4022-ba32-6941d91bfb99-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.227897 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.227816 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8af867d2-6806-4022-ba32-6941d91bfb99-manifests\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.228362 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.228333 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8af867d2-6806-4022-ba32-6941d91bfb99-manifests\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.230207 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.230183 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8af867d2-6806-4022-ba32-6941d91bfb99-webhook-cert\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.230482 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.230461 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8af867d2-6806-4022-ba32-6941d91bfb99-tmp\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.230657 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.230642 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8af867d2-6806-4022-ba32-6941d91bfb99-apiservice-cert\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.235763 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.235742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktfqp\" (UniqueName: \"kubernetes.io/projected/8af867d2-6806-4022-ba32-6941d91bfb99-kube-api-access-ktfqp\") pod \"opendatahub-operator-controller-manager-59c5678768-4d5c9\" (UID: \"8af867d2-6806-4022-ba32-6941d91bfb99\") " pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.303441 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.303378 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:09.431725 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.431694 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9"] Jul 24 01:35:09.434916 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:35:09.434884 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af867d2_6806_4022_ba32_6941d91bfb99.slice/crio-3226bc39dd8267c31370c0cba68580237c717375c9bdc02da2223d53e95e4117 WatchSource:0}: Error finding container 3226bc39dd8267c31370c0cba68580237c717375c9bdc02da2223d53e95e4117: Status 404 returned error can't find the container with id 3226bc39dd8267c31370c0cba68580237c717375c9bdc02da2223d53e95e4117 Jul 24 01:35:09.436726 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:09.436711 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 01:35:10.168094 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:10.168012 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" event={"ID":"8af867d2-6806-4022-ba32-6941d91bfb99","Type":"ContainerStarted","Data":"3226bc39dd8267c31370c0cba68580237c717375c9bdc02da2223d53e95e4117"} Jul 24 01:35:13.180575 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:13.180535 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" event={"ID":"8af867d2-6806-4022-ba32-6941d91bfb99","Type":"ContainerStarted","Data":"52b1fae7020c0f117c39fe2cb3844794437b8188d829ab9f938d026fe5b81f03"} Jul 24 01:35:14.185115 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:14.185085 2564 generic.go:356] "Generic (PLEG): container finished" podID="8af867d2-6806-4022-ba32-6941d91bfb99" containerID="52b1fae7020c0f117c39fe2cb3844794437b8188d829ab9f938d026fe5b81f03" exitCode=0 Jul 24 01:35:14.185497 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:14.185153 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" event={"ID":"8af867d2-6806-4022-ba32-6941d91bfb99","Type":"ContainerDied","Data":"52b1fae7020c0f117c39fe2cb3844794437b8188d829ab9f938d026fe5b81f03"} Jul 24 01:35:15.190609 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:15.190572 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" event={"ID":"8af867d2-6806-4022-ba32-6941d91bfb99","Type":"ContainerStarted","Data":"811a267bb8f8546eeb6587231cf1ab0eb949aa99e1e20b07cdb534e0eb18d537"} Jul 24 01:35:15.190983 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:15.190714 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:15.211681 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:15.211625 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" podStartSLOduration=2.668750792 podStartE2EDuration="7.21161111s" totalImagesPullingTime="3.46308641s" totalInitContainerRuntime="1.079773908s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:35:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:35:15.211355226 +0000 UTC m=+323.676489647" watchObservedRunningTime="2026-07-24 01:35:15.21161111 +0000 UTC m=+323.676745526" Jul 24 01:35:26.196783 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:26.196754 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-59c5678768-4d5c9" Jul 24 01:35:28.721702 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.721667 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8544c7785d-bb94r"] Jul 24 01:35:28.730659 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.730635 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.733100 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.733073 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 01:35:28.734275 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.734255 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 01:35:28.734275 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.734268 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 01:35:28.734449 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.734317 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 01:35:28.734864 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.734847 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9pcv6\"" Jul 24 01:35:28.741551 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.741530 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8544c7785d-bb94r"] Jul 24 01:35:28.866403 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.866368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51487251-cd39-4712-a063-2a7352fdfccb-tls-certs\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.866569 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.866425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51487251-cd39-4712-a063-2a7352fdfccb-tmp\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.866569 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.866476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xnr5\" (UniqueName: \"kubernetes.io/projected/51487251-cd39-4712-a063-2a7352fdfccb-kube-api-access-8xnr5\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.967116 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.967080 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xnr5\" (UniqueName: \"kubernetes.io/projected/51487251-cd39-4712-a063-2a7352fdfccb-kube-api-access-8xnr5\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.967299 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.967155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51487251-cd39-4712-a063-2a7352fdfccb-tls-certs\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.967299 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.967218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51487251-cd39-4712-a063-2a7352fdfccb-tmp\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.969505 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.969479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51487251-cd39-4712-a063-2a7352fdfccb-tmp\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.969661 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.969639 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51487251-cd39-4712-a063-2a7352fdfccb-tls-certs\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:28.975917 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:28.975864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xnr5\" (UniqueName: \"kubernetes.io/projected/51487251-cd39-4712-a063-2a7352fdfccb-kube-api-access-8xnr5\") pod \"kube-auth-proxy-8544c7785d-bb94r\" (UID: \"51487251-cd39-4712-a063-2a7352fdfccb\") " pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:29.039482 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.039442 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" Jul 24 01:35:29.200370 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.200341 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8544c7785d-bb94r"] Jul 24 01:35:29.203928 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:35:29.203895 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51487251_cd39_4712_a063_2a7352fdfccb.slice/crio-534da21a88a5a3df900ae186a2b4e93d1220b1c5c2b18357ef35b2af08a15b3f WatchSource:0}: Error finding container 534da21a88a5a3df900ae186a2b4e93d1220b1c5c2b18357ef35b2af08a15b3f: Status 404 returned error can't find the container with id 534da21a88a5a3df900ae186a2b4e93d1220b1c5c2b18357ef35b2af08a15b3f Jul 24 01:35:29.240054 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.239970 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" event={"ID":"51487251-cd39-4712-a063-2a7352fdfccb","Type":"ContainerStarted","Data":"534da21a88a5a3df900ae186a2b4e93d1220b1c5c2b18357ef35b2af08a15b3f"} Jul 24 01:35:29.697477 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.697446 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs"] Jul 24 01:35:29.701569 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.701547 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.704107 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.704083 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 01:35:29.704261 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.704083 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 01:35:29.704261 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.704244 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tlwvw\"" Jul 24 01:35:29.704491 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.704470 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 01:35:29.704713 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.704697 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 01:35:29.712033 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.712010 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs"] Jul 24 01:35:29.873794 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.873754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.874270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.873804 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.874270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.873825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm9zx\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-kube-api-access-sm9zx\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.874270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.873923 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.874270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.873978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.874270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.874008 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.874270 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.874031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975218 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975218 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975195 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975413 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975413 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975413 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975413 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.975413 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.975374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm9zx\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-kube-api-access-sm9zx\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.977013 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.976985 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.978874 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.978831 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.979360 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.979320 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.979451 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.979377 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.980932 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.980909 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.989518 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.989486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm9zx\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-kube-api-access-sm9zx\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:29.993401 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:29.993364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-f4fzs\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:30.014415 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:30.014357 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:30.151928 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:30.151760 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs"] Jul 24 01:35:30.154699 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:35:30.154671 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4ccb514_f15f_424d_9fbd_870b8bdf7efb.slice/crio-73d692b10dd3315d5a0850fd9e89dccdf8dad20d98f996e888c8d41e8a9ab536 WatchSource:0}: Error finding container 73d692b10dd3315d5a0850fd9e89dccdf8dad20d98f996e888c8d41e8a9ab536: Status 404 returned error can't find the container with id 73d692b10dd3315d5a0850fd9e89dccdf8dad20d98f996e888c8d41e8a9ab536 Jul 24 01:35:30.245611 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:30.245527 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" event={"ID":"e4ccb514-f15f-424d-9fbd-870b8bdf7efb","Type":"ContainerStarted","Data":"73d692b10dd3315d5a0850fd9e89dccdf8dad20d98f996e888c8d41e8a9ab536"} Jul 24 01:35:33.256805 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:33.256771 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 24 01:35:33.257101 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:33.256832 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 24 01:35:34.261206 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.261165 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" event={"ID":"51487251-cd39-4712-a063-2a7352fdfccb","Type":"ContainerStarted","Data":"3e1d4673045c8c3ce7a032424f5fdf39ce8cd7dbe50ce51da3bf1b817fb67c1f"} Jul 24 01:35:34.262687 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.262659 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" event={"ID":"e4ccb514-f15f-424d-9fbd-870b8bdf7efb","Type":"ContainerStarted","Data":"f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f"} Jul 24 01:35:34.262827 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.262793 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:34.264253 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.264225 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-f4fzs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 01:35:34.264374 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.264282 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" podUID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 01:35:34.278917 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.278861 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8544c7785d-bb94r" podStartSLOduration=2.229930786 podStartE2EDuration="6.278845324s" totalImagesPullingTime="4.048914538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:35:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:35:34.277891437 +0000 UTC m=+342.743025870" watchObservedRunningTime="2026-07-24 01:35:34.278845324 +0000 UTC m=+342.743979742" Jul 24 01:35:34.298369 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:34.298309 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" podStartSLOduration=2.19878623 podStartE2EDuration="5.29828766s" totalImagesPullingTime="3.09950143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:35:34.297095809 +0000 UTC m=+342.762230230" watchObservedRunningTime="2026-07-24 01:35:34.29828766 +0000 UTC m=+342.763422078" Jul 24 01:35:35.266729 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:35.266699 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:37.650399 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.650366 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs"] Jul 24 01:35:37.653581 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.653564 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.657948 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.657922 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 01:35:37.657948 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.657937 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-qlfv9\"" Jul 24 01:35:37.658117 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.657925 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 01:35:37.658904 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.658889 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 01:35:37.678100 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.678074 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs"] Jul 24 01:35:37.842100 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.842058 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0555aa78-1095-45dd-94df-85274404aec8-cert\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.842100 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.842101 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0555aa78-1095-45dd-94df-85274404aec8-metrics-cert\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.842365 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.842217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0555aa78-1095-45dd-94df-85274404aec8-manager-config\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.842365 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.842242 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwx4\" (UniqueName: \"kubernetes.io/projected/0555aa78-1095-45dd-94df-85274404aec8-kube-api-access-cpwx4\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.943518 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.943411 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0555aa78-1095-45dd-94df-85274404aec8-manager-config\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.943518 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.943462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwx4\" (UniqueName: \"kubernetes.io/projected/0555aa78-1095-45dd-94df-85274404aec8-kube-api-access-cpwx4\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.943518 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.943506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0555aa78-1095-45dd-94df-85274404aec8-cert\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.943518 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.943524 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0555aa78-1095-45dd-94df-85274404aec8-metrics-cert\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.944194 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.944170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0555aa78-1095-45dd-94df-85274404aec8-manager-config\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.946039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.946016 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0555aa78-1095-45dd-94df-85274404aec8-cert\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.946152 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.946041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0555aa78-1095-45dd-94df-85274404aec8-metrics-cert\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.952590 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.952566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwx4\" (UniqueName: \"kubernetes.io/projected/0555aa78-1095-45dd-94df-85274404aec8-kube-api-access-cpwx4\") pod \"lws-controller-manager-745f9c9fb8-7bpjs\" (UID: \"0555aa78-1095-45dd-94df-85274404aec8\") " pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:37.962250 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:37.962232 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:38.113275 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:38.113251 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs"] Jul 24 01:35:38.114387 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:35:38.114355 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0555aa78_1095_45dd_94df_85274404aec8.slice/crio-1e9226482bfb92dd82eb2cc2cf384f4d63c038b833567993b62d5e33c198c767 WatchSource:0}: Error finding container 1e9226482bfb92dd82eb2cc2cf384f4d63c038b833567993b62d5e33c198c767: Status 404 returned error can't find the container with id 1e9226482bfb92dd82eb2cc2cf384f4d63c038b833567993b62d5e33c198c767 Jul 24 01:35:38.280926 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:38.280846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" event={"ID":"0555aa78-1095-45dd-94df-85274404aec8","Type":"ContainerStarted","Data":"1e9226482bfb92dd82eb2cc2cf384f4d63c038b833567993b62d5e33c198c767"} Jul 24 01:35:40.291685 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:40.291598 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" event={"ID":"0555aa78-1095-45dd-94df-85274404aec8","Type":"ContainerStarted","Data":"6115ee3f9eaf637d892136495ab3e8d96547324cba00c8816aebed00351f64a2"} Jul 24 01:35:40.291685 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:40.291675 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:35:40.320819 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:40.320771 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" podStartSLOduration=1.463971579 podStartE2EDuration="3.320753911s" totalImagesPullingTime="1.856782332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:35:40.317821029 +0000 UTC m=+348.782955456" watchObservedRunningTime="2026-07-24 01:35:40.320753911 +0000 UTC m=+348.785888327" Jul 24 01:35:42.523787 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.523757 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs"] Jul 24 01:35:42.524196 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.523917 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" podUID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" containerName="discovery" containerID="cri-o://f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f" gracePeriod=30 Jul 24 01:35:42.769985 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.769961 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:42.776599 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776541 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-kubeconfig\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.776599 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776571 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-token\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.776599 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776595 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-dns-cert\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.776820 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776613 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm9zx\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-kube-api-access-sm9zx\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.776820 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776635 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-cacerts\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.776820 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776659 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-ca-configmap\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.776820 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.776723 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-local-certs\") pod \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\" (UID: \"e4ccb514-f15f-424d-9fbd-870b8bdf7efb\") " Jul 24 01:35:42.777141 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.777070 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:35:42.778970 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.778942 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 01:35:42.779083 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.779025 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-kube-api-access-sm9zx" (OuterVolumeSpecName: "kube-api-access-sm9zx") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "kube-api-access-sm9zx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:35:42.779279 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.779251 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 01:35:42.779279 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.779257 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-local-certs" (OuterVolumeSpecName: "local-certs") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 01:35:42.779405 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.779303 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:35:42.779405 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.779320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-cacerts" (OuterVolumeSpecName: "cacerts") pod "e4ccb514-f15f-424d-9fbd-870b8bdf7efb" (UID: "e4ccb514-f15f-424d-9fbd-870b8bdf7efb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 01:35:42.878051 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878021 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-local-certs\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:42.878051 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878048 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-kubeconfig\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:42.878274 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878061 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-token\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:42.878274 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878073 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-dns-cert\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:42.878274 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878085 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm9zx\" (UniqueName: \"kubernetes.io/projected/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-kube-api-access-sm9zx\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:42.878274 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878097 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-cacerts\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:42.878274 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:42.878109 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4ccb514-f15f-424d-9fbd-870b8bdf7efb-istio-csr-ca-configmap\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:35:43.303666 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.303634 2564 generic.go:356] "Generic (PLEG): container finished" podID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" containerID="f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f" exitCode=0 Jul 24 01:35:43.303826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.303674 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" Jul 24 01:35:43.303826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.303710 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" event={"ID":"e4ccb514-f15f-424d-9fbd-870b8bdf7efb","Type":"ContainerDied","Data":"f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f"} Jul 24 01:35:43.303826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.303761 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs" event={"ID":"e4ccb514-f15f-424d-9fbd-870b8bdf7efb","Type":"ContainerDied","Data":"73d692b10dd3315d5a0850fd9e89dccdf8dad20d98f996e888c8d41e8a9ab536"} Jul 24 01:35:43.303826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.303782 2564 scope.go:117] "RemoveContainer" containerID="f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f" Jul 24 01:35:43.311914 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.311897 2564 scope.go:117] "RemoveContainer" containerID="f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f" Jul 24 01:35:43.312198 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:35:43.312174 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f\": container with ID starting with f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f not found: ID does not exist" containerID="f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f" Jul 24 01:35:43.312253 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.312210 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f"} err="failed to get container status \"f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f\": rpc error: code = NotFound desc = could not find container \"f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f\": container with ID starting with f6d3792fa327f94ce2d271512c6bd8983b68eeef20dd1266cdf380e40f4aef7f not found: ID does not exist" Jul 24 01:35:43.324381 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.324355 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs"] Jul 24 01:35:43.326628 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.326607 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-f4fzs"] Jul 24 01:35:43.992453 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:43.992357 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" path="/var/lib/kubelet/pods/e4ccb514-f15f-424d-9fbd-870b8bdf7efb/volumes" Jul 24 01:35:51.297273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:35:51.297242 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-745f9c9fb8-7bpjs" Jul 24 01:37:15.741991 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.741906 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h"] Jul 24 01:37:15.742561 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.742193 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" containerName="discovery" Jul 24 01:37:15.742561 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.742206 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" containerName="discovery" Jul 24 01:37:15.742561 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.742258 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4ccb514-f15f-424d-9fbd-870b8bdf7efb" containerName="discovery" Jul 24 01:37:15.744953 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.744937 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.747466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.747440 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 24 01:37:15.747466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.747462 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 01:37:15.747692 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.747540 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 24 01:37:15.747692 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.747562 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 01:37:15.748577 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.748561 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5dwhk\"" Jul 24 01:37:15.753456 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.753436 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h"] Jul 24 01:37:15.759934 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.759913 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cab51544-1b3f-4af6-9b50-9bf2059aa1af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.760015 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.759947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cab51544-1b3f-4af6-9b50-9bf2059aa1af-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.760059 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.760016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqf6k\" (UniqueName: \"kubernetes.io/projected/cab51544-1b3f-4af6-9b50-9bf2059aa1af-kube-api-access-vqf6k\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.860923 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.860877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cab51544-1b3f-4af6-9b50-9bf2059aa1af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.861162 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.860937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cab51544-1b3f-4af6-9b50-9bf2059aa1af-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.861162 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.861002 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqf6k\" (UniqueName: \"kubernetes.io/projected/cab51544-1b3f-4af6-9b50-9bf2059aa1af-kube-api-access-vqf6k\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.861162 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:37:15.861028 2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 24 01:37:15.861162 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:37:15.861110 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab51544-1b3f-4af6-9b50-9bf2059aa1af-plugin-serving-cert podName:cab51544-1b3f-4af6-9b50-9bf2059aa1af nodeName:}" failed. No retries permitted until 2026-07-24 01:37:16.361090322 +0000 UTC m=+444.826224737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cab51544-1b3f-4af6-9b50-9bf2059aa1af-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-kcw6h" (UID: "cab51544-1b3f-4af6-9b50-9bf2059aa1af") : secret "plugin-serving-cert" not found Jul 24 01:37:15.861672 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.861652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cab51544-1b3f-4af6-9b50-9bf2059aa1af-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:15.871093 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:15.871062 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqf6k\" (UniqueName: \"kubernetes.io/projected/cab51544-1b3f-4af6-9b50-9bf2059aa1af-kube-api-access-vqf6k\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:16.364418 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:16.364385 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cab51544-1b3f-4af6-9b50-9bf2059aa1af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:16.366724 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:16.366694 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cab51544-1b3f-4af6-9b50-9bf2059aa1af-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kcw6h\" (UID: \"cab51544-1b3f-4af6-9b50-9bf2059aa1af\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:16.654178 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:16.654157 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" Jul 24 01:37:16.793277 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:16.790463 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h"] Jul 24 01:37:16.794948 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:37:16.794918 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcab51544_1b3f_4af6_9b50_9bf2059aa1af.slice/crio-623b0af6a78bf4c017bf6e36b4601bc6d3e78400d18df51f447497636a1c2549 WatchSource:0}: Error finding container 623b0af6a78bf4c017bf6e36b4601bc6d3e78400d18df51f447497636a1c2549: Status 404 returned error can't find the container with id 623b0af6a78bf4c017bf6e36b4601bc6d3e78400d18df51f447497636a1c2549 Jul 24 01:37:17.637055 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:17.637013 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" event={"ID":"cab51544-1b3f-4af6-9b50-9bf2059aa1af","Type":"ContainerStarted","Data":"623b0af6a78bf4c017bf6e36b4601bc6d3e78400d18df51f447497636a1c2549"} Jul 24 01:37:37.721283 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:37.721237 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" event={"ID":"cab51544-1b3f-4af6-9b50-9bf2059aa1af","Type":"ContainerStarted","Data":"d31d3396730314d0100f5e22bd00ffffb0c29a3ff6d552af4db38f0194cefeac"} Jul 24 01:37:37.741969 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:37:37.741918 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kcw6h" podStartSLOduration=1.932536508 podStartE2EDuration="22.741903691s" totalImagesPullingTime="20.809367183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:37:37.740855316 +0000 UTC m=+466.205989732" watchObservedRunningTime="2026-07-24 01:37:37.741903691 +0000 UTC m=+466.207038107" Jul 24 01:38:16.477657 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.477623 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-wfh2r"] Jul 24 01:38:16.480561 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.480531 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:16.482834 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.482814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sljwb\"" Jul 24 01:38:16.486101 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.486079 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wfh2r"] Jul 24 01:38:16.600556 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.600524 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qfggm"] Jul 24 01:38:16.603532 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.603515 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:16.608330 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.608304 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qfggm"] Jul 24 01:38:16.637379 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.637356 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsz2g\" (UniqueName: \"kubernetes.io/projected/d3ed1416-42e2-4538-9ecc-539e63b4ebd6-kube-api-access-vsz2g\") pod \"authorino-58cd955675-wfh2r\" (UID: \"d3ed1416-42e2-4538-9ecc-539e63b4ebd6\") " pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:16.737884 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.737809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjdns\" (UniqueName: \"kubernetes.io/projected/4ab467b3-78f4-4d89-b345-3b16e26c6825-kube-api-access-sjdns\") pod \"authorino-75d556bd77-qfggm\" (UID: \"4ab467b3-78f4-4d89-b345-3b16e26c6825\") " pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:16.737884 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.737860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsz2g\" (UniqueName: \"kubernetes.io/projected/d3ed1416-42e2-4538-9ecc-539e63b4ebd6-kube-api-access-vsz2g\") pod \"authorino-58cd955675-wfh2r\" (UID: \"d3ed1416-42e2-4538-9ecc-539e63b4ebd6\") " pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:16.745409 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.745386 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsz2g\" (UniqueName: \"kubernetes.io/projected/d3ed1416-42e2-4538-9ecc-539e63b4ebd6-kube-api-access-vsz2g\") pod \"authorino-58cd955675-wfh2r\" (UID: \"d3ed1416-42e2-4538-9ecc-539e63b4ebd6\") " pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:16.790483 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.790457 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:16.838805 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.838772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjdns\" (UniqueName: \"kubernetes.io/projected/4ab467b3-78f4-4d89-b345-3b16e26c6825-kube-api-access-sjdns\") pod \"authorino-75d556bd77-qfggm\" (UID: \"4ab467b3-78f4-4d89-b345-3b16e26c6825\") " pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:16.848043 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.848021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjdns\" (UniqueName: \"kubernetes.io/projected/4ab467b3-78f4-4d89-b345-3b16e26c6825-kube-api-access-sjdns\") pod \"authorino-75d556bd77-qfggm\" (UID: \"4ab467b3-78f4-4d89-b345-3b16e26c6825\") " pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:16.913044 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.913008 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:16.915843 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:16.915694 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-wfh2r"] Jul 24 01:38:16.918664 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:38:16.918640 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3ed1416_42e2_4538_9ecc_539e63b4ebd6.slice/crio-aba7dc6c2da36fd76688431859472ba30a9eaf9c62ff54dda56fbf064b92838f WatchSource:0}: Error finding container aba7dc6c2da36fd76688431859472ba30a9eaf9c62ff54dda56fbf064b92838f: Status 404 returned error can't find the container with id aba7dc6c2da36fd76688431859472ba30a9eaf9c62ff54dda56fbf064b92838f Jul 24 01:38:17.035615 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:17.035591 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qfggm"] Jul 24 01:38:17.037874 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:38:17.037851 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ab467b3_78f4_4d89_b345_3b16e26c6825.slice/crio-60afa6461729351d839c5a8f7f0ad15e7c5a31f511a9328dac8b6a8eb3cc57a0 WatchSource:0}: Error finding container 60afa6461729351d839c5a8f7f0ad15e7c5a31f511a9328dac8b6a8eb3cc57a0: Status 404 returned error can't find the container with id 60afa6461729351d839c5a8f7f0ad15e7c5a31f511a9328dac8b6a8eb3cc57a0 Jul 24 01:38:17.874405 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:17.874368 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qfggm" event={"ID":"4ab467b3-78f4-4d89-b345-3b16e26c6825","Type":"ContainerStarted","Data":"60afa6461729351d839c5a8f7f0ad15e7c5a31f511a9328dac8b6a8eb3cc57a0"} Jul 24 01:38:17.876629 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:17.876552 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wfh2r" event={"ID":"d3ed1416-42e2-4538-9ecc-539e63b4ebd6","Type":"ContainerStarted","Data":"aba7dc6c2da36fd76688431859472ba30a9eaf9c62ff54dda56fbf064b92838f"} Jul 24 01:38:20.891265 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:20.891224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wfh2r" event={"ID":"d3ed1416-42e2-4538-9ecc-539e63b4ebd6","Type":"ContainerStarted","Data":"9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90"} Jul 24 01:38:20.892493 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:20.892463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qfggm" event={"ID":"4ab467b3-78f4-4d89-b345-3b16e26c6825","Type":"ContainerStarted","Data":"993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005"} Jul 24 01:38:20.908418 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:20.908369 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-wfh2r" podStartSLOduration=1.377593524 podStartE2EDuration="4.908355255s" totalImagesPullingTime="3.530761731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:38:20.90733526 +0000 UTC m=+509.372469887" watchObservedRunningTime="2026-07-24 01:38:20.908355255 +0000 UTC m=+509.373489672" Jul 24 01:38:20.923547 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:20.923499 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qfggm" podStartSLOduration=1.5219801039999998 podStartE2EDuration="4.923482438s" totalImagesPullingTime="3.401502334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:38:20.922071455 +0000 UTC m=+509.387205872" watchObservedRunningTime="2026-07-24 01:38:20.923482438 +0000 UTC m=+509.388616855" Jul 24 01:38:20.948956 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:20.948924 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wfh2r"] Jul 24 01:38:22.900098 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:22.900058 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-wfh2r" podUID="d3ed1416-42e2-4538-9ecc-539e63b4ebd6" containerName="authorino" containerID="cri-o://9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90" gracePeriod=30 Jul 24 01:38:23.137591 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.137563 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:23.284608 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.284525 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsz2g\" (UniqueName: \"kubernetes.io/projected/d3ed1416-42e2-4538-9ecc-539e63b4ebd6-kube-api-access-vsz2g\") pod \"d3ed1416-42e2-4538-9ecc-539e63b4ebd6\" (UID: \"d3ed1416-42e2-4538-9ecc-539e63b4ebd6\") " Jul 24 01:38:23.286628 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.286599 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ed1416-42e2-4538-9ecc-539e63b4ebd6-kube-api-access-vsz2g" (OuterVolumeSpecName: "kube-api-access-vsz2g") pod "d3ed1416-42e2-4538-9ecc-539e63b4ebd6" (UID: "d3ed1416-42e2-4538-9ecc-539e63b4ebd6"). InnerVolumeSpecName "kube-api-access-vsz2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:38:23.385033 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.384998 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsz2g\" (UniqueName: \"kubernetes.io/projected/d3ed1416-42e2-4538-9ecc-539e63b4ebd6-kube-api-access-vsz2g\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:38:23.905536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.905500 2564 generic.go:356] "Generic (PLEG): container finished" podID="d3ed1416-42e2-4538-9ecc-539e63b4ebd6" containerID="9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90" exitCode=0 Jul 24 01:38:23.905536 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.905538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wfh2r" event={"ID":"d3ed1416-42e2-4538-9ecc-539e63b4ebd6","Type":"ContainerDied","Data":"9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90"} Jul 24 01:38:23.906046 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.905543 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-wfh2r" Jul 24 01:38:23.906046 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.905564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-wfh2r" event={"ID":"d3ed1416-42e2-4538-9ecc-539e63b4ebd6","Type":"ContainerDied","Data":"aba7dc6c2da36fd76688431859472ba30a9eaf9c62ff54dda56fbf064b92838f"} Jul 24 01:38:23.906046 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.905590 2564 scope.go:117] "RemoveContainer" containerID="9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90" Jul 24 01:38:23.913729 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.913525 2564 scope.go:117] "RemoveContainer" containerID="9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90" Jul 24 01:38:23.913862 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:38:23.913837 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90\": container with ID starting with 9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90 not found: ID does not exist" containerID="9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90" Jul 24 01:38:23.913926 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.913878 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90"} err="failed to get container status \"9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90\": rpc error: code = NotFound desc = could not find container \"9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90\": container with ID starting with 9fe96e4e21a8bae3e2ea542d176253888e2cfbdb039d0f2f5f8cda09a10b3c90 not found: ID does not exist" Jul 24 01:38:23.925191 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.925172 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-wfh2r"] Jul 24 01:38:23.928021 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.928003 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-wfh2r"] Jul 24 01:38:23.991682 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:23.991657 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ed1416-42e2-4538-9ecc-539e63b4ebd6" path="/var/lib/kubelet/pods/d3ed1416-42e2-4538-9ecc-539e63b4ebd6/volumes" Jul 24 01:38:50.125919 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.125886 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gz4dl"] Jul 24 01:38:50.126426 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.126161 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ed1416-42e2-4538-9ecc-539e63b4ebd6" containerName="authorino" Jul 24 01:38:50.126426 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.126173 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ed1416-42e2-4538-9ecc-539e63b4ebd6" containerName="authorino" Jul 24 01:38:50.126426 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.126239 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ed1416-42e2-4538-9ecc-539e63b4ebd6" containerName="authorino" Jul 24 01:38:50.129048 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.129032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:50.136167 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.136143 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gz4dl"] Jul 24 01:38:50.170997 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.170967 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dglvn\" (UniqueName: \"kubernetes.io/projected/0c853da0-360a-4e52-8111-3204e100c5c4-kube-api-access-dglvn\") pod \"authorino-5d4b67c446-gz4dl\" (UID: \"0c853da0-360a-4e52-8111-3204e100c5c4\") " pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:50.271358 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.271319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dglvn\" (UniqueName: \"kubernetes.io/projected/0c853da0-360a-4e52-8111-3204e100c5c4-kube-api-access-dglvn\") pod \"authorino-5d4b67c446-gz4dl\" (UID: \"0c853da0-360a-4e52-8111-3204e100c5c4\") " pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:50.281365 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.281339 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dglvn\" (UniqueName: \"kubernetes.io/projected/0c853da0-360a-4e52-8111-3204e100c5c4-kube-api-access-dglvn\") pod \"authorino-5d4b67c446-gz4dl\" (UID: \"0c853da0-360a-4e52-8111-3204e100c5c4\") " pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:50.302238 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.302208 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gz4dl"] Jul 24 01:38:50.302426 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.302414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:50.328821 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.328794 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-r7xzc"] Jul 24 01:38:50.332119 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.332103 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.334666 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.334647 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 01:38:50.339260 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.339151 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-r7xzc"] Jul 24 01:38:50.371765 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.371705 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff9h6\" (UniqueName: \"kubernetes.io/projected/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-kube-api-access-ff9h6\") pod \"authorino-76877994-r7xzc\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.371916 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.371871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-tls-cert\") pod \"authorino-76877994-r7xzc\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.386432 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.386408 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-r7xzc"] Jul 24 01:38:50.386662 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:38:50.386637 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ff9h6 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-r7xzc" podUID="d40157b8-e2c5-4886-b1fa-fe4f0fbf788b" Jul 24 01:38:50.412556 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.412528 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b68748789-9s25v"] Jul 24 01:38:50.415772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.415750 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.422444 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.422334 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b68748789-9s25v"] Jul 24 01:38:50.437555 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.437529 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gz4dl"] Jul 24 01:38:50.440534 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:38:50.440511 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c853da0_360a_4e52_8111_3204e100c5c4.slice/crio-5a5147348f25d429f326f3eba3a448169608f98cb1bd1c7ed582c84b3bf6c292 WatchSource:0}: Error finding container 5a5147348f25d429f326f3eba3a448169608f98cb1bd1c7ed582c84b3bf6c292: Status 404 returned error can't find the container with id 5a5147348f25d429f326f3eba3a448169608f98cb1bd1c7ed582c84b3bf6c292 Jul 24 01:38:50.472847 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.472812 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-tls-cert\") pod \"authorino-76877994-r7xzc\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.472962 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.472857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-tls-cert\") pod \"authorino-b68748789-9s25v\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.473014 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.472969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lnsv\" (UniqueName: \"kubernetes.io/projected/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-kube-api-access-7lnsv\") pod \"authorino-b68748789-9s25v\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.473067 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.473014 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff9h6\" (UniqueName: \"kubernetes.io/projected/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-kube-api-access-ff9h6\") pod \"authorino-76877994-r7xzc\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.475239 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.475220 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-tls-cert\") pod \"authorino-76877994-r7xzc\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.481373 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.481350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff9h6\" (UniqueName: \"kubernetes.io/projected/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-kube-api-access-ff9h6\") pod \"authorino-76877994-r7xzc\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:50.574060 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.574025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-tls-cert\") pod \"authorino-b68748789-9s25v\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.574060 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.574058 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lnsv\" (UniqueName: \"kubernetes.io/projected/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-kube-api-access-7lnsv\") pod \"authorino-b68748789-9s25v\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.576450 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.576429 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-tls-cert\") pod \"authorino-b68748789-9s25v\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.581685 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.581665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lnsv\" (UniqueName: \"kubernetes.io/projected/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-kube-api-access-7lnsv\") pod \"authorino-b68748789-9s25v\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.725734 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.725653 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:38:50.852319 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:50.852294 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b68748789-9s25v"] Jul 24 01:38:50.854725 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:38:50.854622 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba0ce439_0a01_4e09_a93e_971b5d5d5de5.slice/crio-95459a48388215a2785469b5aa884cdb293bc6c0326a9c7792b55ebfd2ad4866 WatchSource:0}: Error finding container 95459a48388215a2785469b5aa884cdb293bc6c0326a9c7792b55ebfd2ad4866: Status 404 returned error can't find the container with id 95459a48388215a2785469b5aa884cdb293bc6c0326a9c7792b55ebfd2ad4866 Jul 24 01:38:51.004789 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.004697 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b68748789-9s25v" event={"ID":"ba0ce439-0a01-4e09-a93e-971b5d5d5de5","Type":"ContainerStarted","Data":"95459a48388215a2785469b5aa884cdb293bc6c0326a9c7792b55ebfd2ad4866"} Jul 24 01:38:51.006266 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.006235 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" event={"ID":"0c853da0-360a-4e52-8111-3204e100c5c4","Type":"ContainerStarted","Data":"22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc"} Jul 24 01:38:51.006393 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.006271 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" event={"ID":"0c853da0-360a-4e52-8111-3204e100c5c4","Type":"ContainerStarted","Data":"5a5147348f25d429f326f3eba3a448169608f98cb1bd1c7ed582c84b3bf6c292"} Jul 24 01:38:51.006393 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.006293 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:51.006393 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.006281 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" podUID="0c853da0-360a-4e52-8111-3204e100c5c4" containerName="authorino" containerID="cri-o://22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc" gracePeriod=30 Jul 24 01:38:51.013868 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.013849 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:51.021488 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.021447 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" podStartSLOduration=0.68084128 podStartE2EDuration="1.021432331s" totalImagesPullingTime="340.591051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:38:51.020358346 +0000 UTC m=+539.485492777" watchObservedRunningTime="2026-07-24 01:38:51.021432331 +0000 UTC m=+539.486566752" Jul 24 01:38:51.078891 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.078851 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff9h6\" (UniqueName: \"kubernetes.io/projected/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-kube-api-access-ff9h6\") pod \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " Jul 24 01:38:51.079068 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.078925 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-tls-cert\") pod \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\" (UID: \"d40157b8-e2c5-4886-b1fa-fe4f0fbf788b\") " Jul 24 01:38:51.081030 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.081008 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d40157b8-e2c5-4886-b1fa-fe4f0fbf788b" (UID: "d40157b8-e2c5-4886-b1fa-fe4f0fbf788b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 01:38:51.081110 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.081054 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-kube-api-access-ff9h6" (OuterVolumeSpecName: "kube-api-access-ff9h6") pod "d40157b8-e2c5-4886-b1fa-fe4f0fbf788b" (UID: "d40157b8-e2c5-4886-b1fa-fe4f0fbf788b"). InnerVolumeSpecName "kube-api-access-ff9h6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:38:51.179783 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.179748 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff9h6\" (UniqueName: \"kubernetes.io/projected/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-kube-api-access-ff9h6\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:38:51.179783 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.179784 2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b-tls-cert\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:38:51.234418 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.234396 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:51.281576 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.281507 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dglvn\" (UniqueName: \"kubernetes.io/projected/0c853da0-360a-4e52-8111-3204e100c5c4-kube-api-access-dglvn\") pod \"0c853da0-360a-4e52-8111-3204e100c5c4\" (UID: \"0c853da0-360a-4e52-8111-3204e100c5c4\") " Jul 24 01:38:51.283531 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.283502 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c853da0-360a-4e52-8111-3204e100c5c4-kube-api-access-dglvn" (OuterVolumeSpecName: "kube-api-access-dglvn") pod "0c853da0-360a-4e52-8111-3204e100c5c4" (UID: "0c853da0-360a-4e52-8111-3204e100c5c4"). InnerVolumeSpecName "kube-api-access-dglvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:38:51.382760 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:51.382717 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dglvn\" (UniqueName: \"kubernetes.io/projected/0c853da0-360a-4e52-8111-3204e100c5c4-kube-api-access-dglvn\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:38:52.011043 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.011011 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b68748789-9s25v" event={"ID":"ba0ce439-0a01-4e09-a93e-971b5d5d5de5","Type":"ContainerStarted","Data":"25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e"} Jul 24 01:38:52.012079 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.012054 2564 generic.go:356] "Generic (PLEG): container finished" podID="0c853da0-360a-4e52-8111-3204e100c5c4" containerID="22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc" exitCode=0 Jul 24 01:38:52.012205 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.012087 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" Jul 24 01:38:52.012205 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.012189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" event={"ID":"0c853da0-360a-4e52-8111-3204e100c5c4","Type":"ContainerDied","Data":"22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc"} Jul 24 01:38:52.012318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.012213 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-r7xzc" Jul 24 01:38:52.012318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.012221 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gz4dl" event={"ID":"0c853da0-360a-4e52-8111-3204e100c5c4","Type":"ContainerDied","Data":"5a5147348f25d429f326f3eba3a448169608f98cb1bd1c7ed582c84b3bf6c292"} Jul 24 01:38:52.012318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.012253 2564 scope.go:117] "RemoveContainer" containerID="22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc" Jul 24 01:38:52.021848 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.021809 2564 scope.go:117] "RemoveContainer" containerID="22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc" Jul 24 01:38:52.022058 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:38:52.022019 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc\": container with ID starting with 22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc not found: ID does not exist" containerID="22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc" Jul 24 01:38:52.022246 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.022064 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc"} err="failed to get container status \"22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc\": rpc error: code = NotFound desc = could not find container \"22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc\": container with ID starting with 22dc4246537bbac39df5223eb3eaeb3343172337fe1be71b87199bcbf2672bdc not found: ID does not exist" Jul 24 01:38:52.029333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.029295 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b68748789-9s25v" podStartSLOduration=1.68838087 podStartE2EDuration="2.029279828s" totalImagesPullingTime="340.898958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:38:52.028553215 +0000 UTC m=+540.493687632" watchObservedRunningTime="2026-07-24 01:38:52.029279828 +0000 UTC m=+540.494414272" Jul 24 01:38:52.053339 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.053307 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-r7xzc"] Jul 24 01:38:52.058763 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.058730 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qfggm"] Jul 24 01:38:52.058910 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.058888 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qfggm" podUID="4ab467b3-78f4-4d89-b345-3b16e26c6825" containerName="authorino" containerID="cri-o://993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005" gracePeriod=30 Jul 24 01:38:52.060710 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.060691 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-r7xzc"] Jul 24 01:38:52.073394 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.073367 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gz4dl"] Jul 24 01:38:52.079175 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.079153 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gz4dl"] Jul 24 01:38:52.301793 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.301771 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:52.390312 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.390284 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjdns\" (UniqueName: \"kubernetes.io/projected/4ab467b3-78f4-4d89-b345-3b16e26c6825-kube-api-access-sjdns\") pod \"4ab467b3-78f4-4d89-b345-3b16e26c6825\" (UID: \"4ab467b3-78f4-4d89-b345-3b16e26c6825\") " Jul 24 01:38:52.392381 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.392352 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab467b3-78f4-4d89-b345-3b16e26c6825-kube-api-access-sjdns" (OuterVolumeSpecName: "kube-api-access-sjdns") pod "4ab467b3-78f4-4d89-b345-3b16e26c6825" (UID: "4ab467b3-78f4-4d89-b345-3b16e26c6825"). InnerVolumeSpecName "kube-api-access-sjdns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:38:52.491263 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:52.491185 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjdns\" (UniqueName: \"kubernetes.io/projected/4ab467b3-78f4-4d89-b345-3b16e26c6825-kube-api-access-sjdns\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:38:53.017583 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.017548 2564 generic.go:356] "Generic (PLEG): container finished" podID="4ab467b3-78f4-4d89-b345-3b16e26c6825" containerID="993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005" exitCode=0 Jul 24 01:38:53.017754 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.017588 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qfggm" Jul 24 01:38:53.017754 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.017623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qfggm" event={"ID":"4ab467b3-78f4-4d89-b345-3b16e26c6825","Type":"ContainerDied","Data":"993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005"} Jul 24 01:38:53.017754 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.017658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qfggm" event={"ID":"4ab467b3-78f4-4d89-b345-3b16e26c6825","Type":"ContainerDied","Data":"60afa6461729351d839c5a8f7f0ad15e7c5a31f511a9328dac8b6a8eb3cc57a0"} Jul 24 01:38:53.017754 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.017680 2564 scope.go:117] "RemoveContainer" containerID="993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005" Jul 24 01:38:53.025698 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.025679 2564 scope.go:117] "RemoveContainer" containerID="993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005" Jul 24 01:38:53.025933 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:38:53.025916 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005\": container with ID starting with 993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005 not found: ID does not exist" containerID="993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005" Jul 24 01:38:53.025991 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.025940 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005"} err="failed to get container status \"993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005\": rpc error: code = NotFound desc = could not find container \"993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005\": container with ID starting with 993276aa8107c96613b7ee560ea004fac7dab47d052dd5537a4239b70fdc6005 not found: ID does not exist" Jul 24 01:38:53.037387 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.037354 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qfggm"] Jul 24 01:38:53.043952 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.043932 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qfggm"] Jul 24 01:38:53.991677 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.991643 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c853da0-360a-4e52-8111-3204e100c5c4" path="/var/lib/kubelet/pods/0c853da0-360a-4e52-8111-3204e100c5c4/volumes" Jul 24 01:38:53.992045 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.991934 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab467b3-78f4-4d89-b345-3b16e26c6825" path="/var/lib/kubelet/pods/4ab467b3-78f4-4d89-b345-3b16e26c6825/volumes" Jul 24 01:38:53.992243 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:38:53.992230 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d40157b8-e2c5-4886-b1fa-fe4f0fbf788b" path="/var/lib/kubelet/pods/d40157b8-e2c5-4886-b1fa-fe4f0fbf788b/volumes" Jul 24 01:39:31.696865 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.696832 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8"] Jul 24 01:39:31.697388 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.697138 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab467b3-78f4-4d89-b345-3b16e26c6825" containerName="authorino" Jul 24 01:39:31.697388 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.697155 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab467b3-78f4-4d89-b345-3b16e26c6825" containerName="authorino" Jul 24 01:39:31.697388 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.697163 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c853da0-360a-4e52-8111-3204e100c5c4" containerName="authorino" Jul 24 01:39:31.697388 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.697168 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c853da0-360a-4e52-8111-3204e100c5c4" containerName="authorino" Jul 24 01:39:31.697388 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.697214 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c853da0-360a-4e52-8111-3204e100c5c4" containerName="authorino" Jul 24 01:39:31.697388 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.697230 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab467b3-78f4-4d89-b345-3b16e26c6825" containerName="authorino" Jul 24 01:39:31.699844 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.699828 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:31.702512 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.702489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 01:39:31.711506 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.711486 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8"] Jul 24 01:39:31.765667 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.765638 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qsm4\" (UniqueName: \"kubernetes.io/projected/c02b265e-5e03-4861-af73-fb3b3f96d129-kube-api-access-7qsm4\") pod \"payload-pre-processing-687dbb6cf6-xdsj8\" (UID: \"c02b265e-5e03-4861-af73-fb3b3f96d129\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:31.765824 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.765682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02b265e-5e03-4861-af73-fb3b3f96d129-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-xdsj8\" (UID: \"c02b265e-5e03-4861-af73-fb3b3f96d129\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:31.866218 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.866178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qsm4\" (UniqueName: \"kubernetes.io/projected/c02b265e-5e03-4861-af73-fb3b3f96d129-kube-api-access-7qsm4\") pod \"payload-pre-processing-687dbb6cf6-xdsj8\" (UID: \"c02b265e-5e03-4861-af73-fb3b3f96d129\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:31.866218 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.866220 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02b265e-5e03-4861-af73-fb3b3f96d129-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-xdsj8\" (UID: \"c02b265e-5e03-4861-af73-fb3b3f96d129\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:31.866893 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.866866 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c02b265e-5e03-4861-af73-fb3b3f96d129-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-xdsj8\" (UID: \"c02b265e-5e03-4861-af73-fb3b3f96d129\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:31.873955 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:31.873926 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qsm4\" (UniqueName: \"kubernetes.io/projected/c02b265e-5e03-4861-af73-fb3b3f96d129-kube-api-access-7qsm4\") pod \"payload-pre-processing-687dbb6cf6-xdsj8\" (UID: \"c02b265e-5e03-4861-af73-fb3b3f96d129\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:32.009856 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:32.009777 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:32.137611 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:32.137574 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8"] Jul 24 01:39:32.140485 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:39:32.140456 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02b265e_5e03_4861_af73_fb3b3f96d129.slice/crio-b2a91fb64ae550387f0beb97f4bfa514f416dc8e9a437becbc61d97c6714ce1e WatchSource:0}: Error finding container b2a91fb64ae550387f0beb97f4bfa514f416dc8e9a437becbc61d97c6714ce1e: Status 404 returned error can't find the container with id b2a91fb64ae550387f0beb97f4bfa514f416dc8e9a437becbc61d97c6714ce1e Jul 24 01:39:32.171255 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:32.171224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" event={"ID":"c02b265e-5e03-4861-af73-fb3b3f96d129","Type":"ContainerStarted","Data":"b2a91fb64ae550387f0beb97f4bfa514f416dc8e9a437becbc61d97c6714ce1e"} Jul 24 01:39:35.186193 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:35.186156 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" event={"ID":"c02b265e-5e03-4861-af73-fb3b3f96d129","Type":"ContainerStarted","Data":"1687b4a61cd11e7a699e44458f7c73b5214bf8d256ab57d9437ee1107249fdc4"} Jul 24 01:39:35.186579 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:35.186240 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:35.200061 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:35.199954 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" podStartSLOduration=1.860628668 podStartE2EDuration="4.199934878s" totalImagesPullingTime="2.33930621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:39:35.199818811 +0000 UTC m=+583.664953252" watchObservedRunningTime="2026-07-24 01:39:35.199934878 +0000 UTC m=+583.665069297" Jul 24 01:39:46.193039 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:46.193004 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-xdsj8" Jul 24 01:39:51.841148 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:51.841103 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:39:51.841682 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:39:51.841197 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:40:05.564845 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.564811 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4"] Jul 24 01:40:05.569281 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.569265 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.571987 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.571968 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 01:40:05.573242 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.573223 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 24 01:40:05.573482 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.573468 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 01:40:05.573578 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.573563 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bqxvq\"" Jul 24 01:40:05.579154 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.579113 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4"] Jul 24 01:40:05.610567 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.610534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thdqm\" (UniqueName: \"kubernetes.io/projected/959f462b-47be-44e3-9047-2191c1771d7e-kube-api-access-thdqm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.610665 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.610573 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.610665 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.610597 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959f462b-47be-44e3-9047-2191c1771d7e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.610755 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.610664 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.610755 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.610705 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.610755 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.610736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.711945 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.711917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712092 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.711960 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thdqm\" (UniqueName: \"kubernetes.io/projected/959f462b-47be-44e3-9047-2191c1771d7e-kube-api-access-thdqm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712092 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.711984 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712092 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.712003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959f462b-47be-44e3-9047-2191c1771d7e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712276 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.712119 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712276 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.712200 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712423 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.712392 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712548 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.712450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.712610 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.712555 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.714268 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.714248 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959f462b-47be-44e3-9047-2191c1771d7e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.714435 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.714418 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959f462b-47be-44e3-9047-2191c1771d7e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.720173 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.720120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thdqm\" (UniqueName: \"kubernetes.io/projected/959f462b-47be-44e3-9047-2191c1771d7e-kube-api-access-thdqm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vx8z4\" (UID: \"959f462b-47be-44e3-9047-2191c1771d7e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:05.881291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:05.881257 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:06.007275 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:06.007253 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4"] Jul 24 01:40:06.009580 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:40:06.009550 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959f462b_47be_44e3_9047_2191c1771d7e.slice/crio-af029b78b6d7916488e711af859888e08003b2d942cacd25b3b2d786762a8a66 WatchSource:0}: Error finding container af029b78b6d7916488e711af859888e08003b2d942cacd25b3b2d786762a8a66: Status 404 returned error can't find the container with id af029b78b6d7916488e711af859888e08003b2d942cacd25b3b2d786762a8a66 Jul 24 01:40:06.307024 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:06.306940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" event={"ID":"959f462b-47be-44e3-9047-2191c1771d7e","Type":"ContainerStarted","Data":"af029b78b6d7916488e711af859888e08003b2d942cacd25b3b2d786762a8a66"} Jul 24 01:40:07.312764 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:07.312731 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" event={"ID":"959f462b-47be-44e3-9047-2191c1771d7e","Type":"ContainerStarted","Data":"5c51279c9b3aa8022d02e030baa2dc1af18bd9a5dd905b4b324fadff26790449"} Jul 24 01:40:07.331068 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:07.331023 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" podStartSLOduration=1.111732975 podStartE2EDuration="2.331008161s" totalImagesPullingTime="1.219275186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:40:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:40:07.329385432 +0000 UTC m=+615.794519878" watchObservedRunningTime="2026-07-24 01:40:07.331008161 +0000 UTC m=+615.796142578" Jul 24 01:40:15.882444 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:15.882364 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:15.882444 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:15.882422 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:15.894722 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:15.894702 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:16.361699 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:16.361620 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vx8z4" Jul 24 01:40:19.867676 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.867640 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4"] Jul 24 01:40:19.870887 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.870863 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:19.873417 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.873397 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 24 01:40:19.880216 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.880195 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4"] Jul 24 01:40:19.926140 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.926083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:19.926140 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.926121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c327dc22-9d25-4072-9dcb-3b04228a12ab-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:19.926358 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.926212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pdf4\" (UniqueName: \"kubernetes.io/projected/c327dc22-9d25-4072-9dcb-3b04228a12ab-kube-api-access-7pdf4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:19.926358 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.926243 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:19.926358 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.926273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:19.926358 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:19.926308 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.026707 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.026665 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pdf4\" (UniqueName: \"kubernetes.io/projected/c327dc22-9d25-4072-9dcb-3b04228a12ab-kube-api-access-7pdf4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.026707 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.026707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.026959 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.026734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.026959 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.026755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.026959 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.026781 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.026959 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.026799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c327dc22-9d25-4072-9dcb-3b04228a12ab-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.027209 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.027190 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.027272 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.027249 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.027319 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.027273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.029156 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.029117 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c327dc22-9d25-4072-9dcb-3b04228a12ab-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.029529 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.029508 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c327dc22-9d25-4072-9dcb-3b04228a12ab-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.034189 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.034165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pdf4\" (UniqueName: \"kubernetes.io/projected/c327dc22-9d25-4072-9dcb-3b04228a12ab-kube-api-access-7pdf4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4\" (UID: \"c327dc22-9d25-4072-9dcb-3b04228a12ab\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.181067 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.181036 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:20.306935 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.306913 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4"] Jul 24 01:40:20.309454 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:40:20.309421 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc327dc22_9d25_4072_9dcb_3b04228a12ab.slice/crio-bccfb28f82707d89aa4cee2d94a6a942fcf17ff3395341e2c10b7c6e2287862f WatchSource:0}: Error finding container bccfb28f82707d89aa4cee2d94a6a942fcf17ff3395341e2c10b7c6e2287862f: Status 404 returned error can't find the container with id bccfb28f82707d89aa4cee2d94a6a942fcf17ff3395341e2c10b7c6e2287862f Jul 24 01:40:20.310838 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.310820 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 01:40:20.366106 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:20.366073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" event={"ID":"c327dc22-9d25-4072-9dcb-3b04228a12ab","Type":"ContainerStarted","Data":"bccfb28f82707d89aa4cee2d94a6a942fcf17ff3395341e2c10b7c6e2287862f"} Jul 24 01:40:21.371511 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:21.371471 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" event={"ID":"c327dc22-9d25-4072-9dcb-3b04228a12ab","Type":"ContainerStarted","Data":"5b204a181494b83476650dcf419eda50f182675cf35f3af1556a1e0b7f884356"} Jul 24 01:40:21.390509 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:21.390448 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" podStartSLOduration=2.190605763 podStartE2EDuration="2.390431741s" totalImagesPullingTime="199.825978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:40:21.388545604 +0000 UTC m=+629.853680022" watchObservedRunningTime="2026-07-24 01:40:21.390431741 +0000 UTC m=+629.855566211" Jul 24 01:40:30.181587 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:30.181552 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:30.182071 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:30.181617 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:30.194169 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:30.194118 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:30.421836 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:30.421807 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4" Jul 24 01:40:32.068936 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.068904 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv"] Jul 24 01:40:32.072333 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.072314 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.074803 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.074779 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 24 01:40:32.083740 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.083719 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv"] Jul 24 01:40:32.127005 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.126966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.127181 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.127033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.127181 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.127055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.127181 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.127079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.127181 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.127108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5xx6\" (UniqueName: \"kubernetes.io/projected/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-kube-api-access-j5xx6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.127181 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.127169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228519 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228519 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228777 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228777 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5xx6\" (UniqueName: \"kubernetes.io/projected/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-kube-api-access-j5xx6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228777 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228777 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228620 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.228981 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.229033 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.228978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.229033 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.229020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.230888 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.230863 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.230999 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.230984 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.236672 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.236648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5xx6\" (UniqueName: \"kubernetes.io/projected/aeb6b40e-5703-4cdf-b54e-58c00e5d2645-kube-api-access-j5xx6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lxrdv\" (UID: \"aeb6b40e-5703-4cdf-b54e-58c00e5d2645\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.383287 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.383256 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:32.508866 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:32.508843 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv"] Jul 24 01:40:32.511382 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:40:32.511352 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeb6b40e_5703_4cdf_b54e_58c00e5d2645.slice/crio-2b4875acfaff8cdc5faef81213480542fd4f2ed59d8019aeb029ca3d00cb014f WatchSource:0}: Error finding container 2b4875acfaff8cdc5faef81213480542fd4f2ed59d8019aeb029ca3d00cb014f: Status 404 returned error can't find the container with id 2b4875acfaff8cdc5faef81213480542fd4f2ed59d8019aeb029ca3d00cb014f Jul 24 01:40:33.420294 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:33.420256 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" event={"ID":"aeb6b40e-5703-4cdf-b54e-58c00e5d2645","Type":"ContainerStarted","Data":"971a6aac9ad53d28776dea5505dd083680edd02e632c1a6aaf25b66a43e5b5fa"} Jul 24 01:40:33.420672 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:33.420322 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" event={"ID":"aeb6b40e-5703-4cdf-b54e-58c00e5d2645","Type":"ContainerStarted","Data":"2b4875acfaff8cdc5faef81213480542fd4f2ed59d8019aeb029ca3d00cb014f"} Jul 24 01:40:33.438199 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:33.438139 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" podStartSLOduration=1.232345809 podStartE2EDuration="1.438102976s" totalImagesPullingTime="205.757167ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:40:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:40:33.436688732 +0000 UTC m=+641.901823140" watchObservedRunningTime="2026-07-24 01:40:33.438102976 +0000 UTC m=+641.903237395" Jul 24 01:40:35.773138 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.773086 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4"] Jul 24 01:40:35.780606 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.778585 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.781603 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.781581 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 24 01:40:35.785147 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.785098 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4"] Jul 24 01:40:35.861324 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.861289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.861324 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.861331 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.861543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.861366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.861543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.861390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp5l2\" (UniqueName: \"kubernetes.io/projected/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-kube-api-access-bp5l2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.861543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.861415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.861543 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.861452 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.962630 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.962590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.962630 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.962633 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.962839 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.962671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.962839 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.962699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp5l2\" (UniqueName: \"kubernetes.io/projected/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-kube-api-access-bp5l2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.962839 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.962733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.962839 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.962773 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.963160 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.963110 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.963217 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.963166 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.963275 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.963251 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.965066 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.965042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.965234 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.965215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:35.971476 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:35.971454 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp5l2\" (UniqueName: \"kubernetes.io/projected/b2e3536d-82fa-4483-9695-4c3eeeac1ab1-kube-api-access-bp5l2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4\" (UID: \"b2e3536d-82fa-4483-9695-4c3eeeac1ab1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:36.092756 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:36.092670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:36.221663 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:36.221638 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4"] Jul 24 01:40:36.222531 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:40:36.222504 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2e3536d_82fa_4483_9695_4c3eeeac1ab1.slice/crio-198d161def2c3b1c6d2925d0249d32f2d3e07abb1a6ce3fe5018cd8c714ff2a2 WatchSource:0}: Error finding container 198d161def2c3b1c6d2925d0249d32f2d3e07abb1a6ce3fe5018cd8c714ff2a2: Status 404 returned error can't find the container with id 198d161def2c3b1c6d2925d0249d32f2d3e07abb1a6ce3fe5018cd8c714ff2a2 Jul 24 01:40:36.434725 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:36.434699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" event={"ID":"b2e3536d-82fa-4483-9695-4c3eeeac1ab1","Type":"ContainerStarted","Data":"198d161def2c3b1c6d2925d0249d32f2d3e07abb1a6ce3fe5018cd8c714ff2a2"} Jul 24 01:40:37.440607 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:37.440575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" event={"ID":"b2e3536d-82fa-4483-9695-4c3eeeac1ab1","Type":"ContainerStarted","Data":"0a39c2ea7c871f830fbd8ecb7780d39e89c217494769b83289cde25ba4384e8e"} Jul 24 01:40:37.458268 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:37.458221 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" podStartSLOduration=2.256987818 podStartE2EDuration="2.458204366s" totalImagesPullingTime="201.216548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:40:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:40:37.457026402 +0000 UTC m=+645.922160815" watchObservedRunningTime="2026-07-24 01:40:37.458204366 +0000 UTC m=+645.923338785" Jul 24 01:40:42.384352 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:42.384318 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:42.384739 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:42.384401 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:42.396805 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:42.396783 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:42.475145 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:42.475101 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lxrdv" Jul 24 01:40:46.093783 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:46.093750 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:46.094190 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:46.093799 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:46.105894 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:46.105871 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:40:46.489337 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:40:46.489303 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4" Jul 24 01:41:41.047746 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.047711 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-758df5668-tdl5s"] Jul 24 01:41:41.050943 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.050913 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.057989 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.057966 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-758df5668-tdl5s"] Jul 24 01:41:41.185382 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.185341 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f235a10-df42-48f2-a41f-b2e2f095bed7-tls-cert\") pod \"authorino-758df5668-tdl5s\" (UID: \"1f235a10-df42-48f2-a41f-b2e2f095bed7\") " pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.185382 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.185376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbbgf\" (UniqueName: \"kubernetes.io/projected/1f235a10-df42-48f2-a41f-b2e2f095bed7-kube-api-access-fbbgf\") pod \"authorino-758df5668-tdl5s\" (UID: \"1f235a10-df42-48f2-a41f-b2e2f095bed7\") " pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.286481 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.286449 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f235a10-df42-48f2-a41f-b2e2f095bed7-tls-cert\") pod \"authorino-758df5668-tdl5s\" (UID: \"1f235a10-df42-48f2-a41f-b2e2f095bed7\") " pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.286481 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.286485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbbgf\" (UniqueName: \"kubernetes.io/projected/1f235a10-df42-48f2-a41f-b2e2f095bed7-kube-api-access-fbbgf\") pod \"authorino-758df5668-tdl5s\" (UID: \"1f235a10-df42-48f2-a41f-b2e2f095bed7\") " pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.288837 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.288806 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f235a10-df42-48f2-a41f-b2e2f095bed7-tls-cert\") pod \"authorino-758df5668-tdl5s\" (UID: \"1f235a10-df42-48f2-a41f-b2e2f095bed7\") " pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.294759 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.294731 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbbgf\" (UniqueName: \"kubernetes.io/projected/1f235a10-df42-48f2-a41f-b2e2f095bed7-kube-api-access-fbbgf\") pod \"authorino-758df5668-tdl5s\" (UID: \"1f235a10-df42-48f2-a41f-b2e2f095bed7\") " pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.359999 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.359937 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758df5668-tdl5s" Jul 24 01:41:41.484523 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.484486 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-758df5668-tdl5s"] Jul 24 01:41:41.488121 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:41:41.488090 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f235a10_df42_48f2_a41f_b2e2f095bed7.slice/crio-a92d806cfc6e5f518f2984e47acca0d4af62fdb8e3f0f439ad4c1cdbe178db49 WatchSource:0}: Error finding container a92d806cfc6e5f518f2984e47acca0d4af62fdb8e3f0f439ad4c1cdbe178db49: Status 404 returned error can't find the container with id a92d806cfc6e5f518f2984e47acca0d4af62fdb8e3f0f439ad4c1cdbe178db49 Jul 24 01:41:41.698055 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:41.698021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758df5668-tdl5s" event={"ID":"1f235a10-df42-48f2-a41f-b2e2f095bed7","Type":"ContainerStarted","Data":"a92d806cfc6e5f518f2984e47acca0d4af62fdb8e3f0f439ad4c1cdbe178db49"} Jul 24 01:41:42.703875 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:42.703838 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758df5668-tdl5s" event={"ID":"1f235a10-df42-48f2-a41f-b2e2f095bed7","Type":"ContainerStarted","Data":"12e926038412609a8bcf71b17ad1891b7ff900514dc1e4a38c317b9845bc922b"} Jul 24 01:41:42.722498 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:42.722443 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-758df5668-tdl5s" podStartSLOduration=1.2484145500000001 podStartE2EDuration="1.722428292s" totalImagesPullingTime="474.013742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:41:42.720582192 +0000 UTC m=+711.185716634" watchObservedRunningTime="2026-07-24 01:41:42.722428292 +0000 UTC m=+711.187562709" Jul 24 01:41:42.748074 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:42.748041 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b68748789-9s25v"] Jul 24 01:41:42.748273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:42.748251 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b68748789-9s25v" podUID="ba0ce439-0a01-4e09-a93e-971b5d5d5de5" containerName="authorino" containerID="cri-o://25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e" gracePeriod=30 Jul 24 01:41:43.005897 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.005871 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:41:43.100233 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.100197 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-tls-cert\") pod \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " Jul 24 01:41:43.100400 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.100264 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lnsv\" (UniqueName: \"kubernetes.io/projected/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-kube-api-access-7lnsv\") pod \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\" (UID: \"ba0ce439-0a01-4e09-a93e-971b5d5d5de5\") " Jul 24 01:41:43.102249 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.102216 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-kube-api-access-7lnsv" (OuterVolumeSpecName: "kube-api-access-7lnsv") pod "ba0ce439-0a01-4e09-a93e-971b5d5d5de5" (UID: "ba0ce439-0a01-4e09-a93e-971b5d5d5de5"). InnerVolumeSpecName "kube-api-access-7lnsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:41:43.109568 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.109537 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ba0ce439-0a01-4e09-a93e-971b5d5d5de5" (UID: "ba0ce439-0a01-4e09-a93e-971b5d5d5de5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 01:41:43.201779 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.201739 2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-tls-cert\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:41:43.201779 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.201779 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lnsv\" (UniqueName: \"kubernetes.io/projected/ba0ce439-0a01-4e09-a93e-971b5d5d5de5-kube-api-access-7lnsv\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:41:43.710196 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.710153 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba0ce439-0a01-4e09-a93e-971b5d5d5de5" containerID="25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e" exitCode=0 Jul 24 01:41:43.710668 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.710209 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b68748789-9s25v" Jul 24 01:41:43.710668 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.710255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b68748789-9s25v" event={"ID":"ba0ce439-0a01-4e09-a93e-971b5d5d5de5","Type":"ContainerDied","Data":"25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e"} Jul 24 01:41:43.710668 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.710301 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b68748789-9s25v" event={"ID":"ba0ce439-0a01-4e09-a93e-971b5d5d5de5","Type":"ContainerDied","Data":"95459a48388215a2785469b5aa884cdb293bc6c0326a9c7792b55ebfd2ad4866"} Jul 24 01:41:43.710668 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.710326 2564 scope.go:117] "RemoveContainer" containerID="25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e" Jul 24 01:41:43.718827 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.718809 2564 scope.go:117] "RemoveContainer" containerID="25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e" Jul 24 01:41:43.719119 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:41:43.719097 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e\": container with ID starting with 25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e not found: ID does not exist" containerID="25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e" Jul 24 01:41:43.719201 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.719169 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e"} err="failed to get container status \"25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e\": rpc error: code = NotFound desc = could not find container \"25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e\": container with ID starting with 25b1fe90889f6cf62c54824a200b89a09e697dd2ba95afb0faf66ffdc6d93b9e not found: ID does not exist" Jul 24 01:41:43.733687 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.733660 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b68748789-9s25v"] Jul 24 01:41:43.748883 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.744964 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b68748789-9s25v"] Jul 24 01:41:43.992506 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:41:43.992377 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba0ce439-0a01-4e09-a93e-971b5d5d5de5" path="/var/lib/kubelet/pods/ba0ce439-0a01-4e09-a93e-971b5d5d5de5/volumes" Jul 24 01:44:51.864797 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:44:51.864763 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:44:51.866641 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:44:51.866617 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:49:51.888778 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:49:51.888745 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:49:51.890031 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:49:51.890009 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:54:51.910639 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:54:51.910611 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:54:51.912232 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:54:51.912211 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:56:52.449118 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.449073 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx"] Jul 24 01:56:52.450986 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.449506 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0ce439-0a01-4e09-a93e-971b5d5d5de5" containerName="authorino" Jul 24 01:56:52.450986 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.449521 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0ce439-0a01-4e09-a93e-971b5d5d5de5" containerName="authorino" Jul 24 01:56:52.450986 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.449573 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba0ce439-0a01-4e09-a93e-971b5d5d5de5" containerName="authorino" Jul 24 01:56:52.452310 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.452295 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.455018 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.454998 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-44fb0de5-dockercfg-x8542\"" Jul 24 01:56:52.455147 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.455040 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-44fb0de5\"" Jul 24 01:56:52.462947 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.462923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx"] Jul 24 01:56:52.549942 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.549911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhtln\" (UniqueName: \"kubernetes.io/projected/a02fc955-95ba-4773-b629-92390c08de2d-kube-api-access-nhtln\") pod \"payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.550111 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.549950 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a02fc955-95ba-4773-b629-92390c08de2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.651101 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.651060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhtln\" (UniqueName: \"kubernetes.io/projected/a02fc955-95ba-4773-b629-92390c08de2d-kube-api-access-nhtln\") pod \"payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.651289 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.651110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a02fc955-95ba-4773-b629-92390c08de2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.651879 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.651857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a02fc955-95ba-4773-b629-92390c08de2d-plugins-config-volume\") pod \"payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.659485 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.659459 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhtln\" (UniqueName: \"kubernetes.io/projected/a02fc955-95ba-4773-b629-92390c08de2d-kube-api-access-nhtln\") pod \"payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.764304 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.764208 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:52.898109 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:52.898083 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx"] Jul 24 01:56:52.900675 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:56:52.900637 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02fc955_95ba_4773_b629_92390c08de2d.slice/crio-cbad3ad05915fa382a6565d4578c57b3aa382b8b0a52c8dc810a901939785ca6 WatchSource:0}: Error finding container cbad3ad05915fa382a6565d4578c57b3aa382b8b0a52c8dc810a901939785ca6: Status 404 returned error can't find the container with id cbad3ad05915fa382a6565d4578c57b3aa382b8b0a52c8dc810a901939785ca6 Jul 24 01:56:53.415318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:53.415279 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" event={"ID":"a02fc955-95ba-4773-b629-92390c08de2d","Type":"ContainerStarted","Data":"c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d"} Jul 24 01:56:53.415513 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:53.415332 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" event={"ID":"a02fc955-95ba-4773-b629-92390c08de2d","Type":"ContainerStarted","Data":"cbad3ad05915fa382a6565d4578c57b3aa382b8b0a52c8dc810a901939785ca6"} Jul 24 01:56:53.415513 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:53.415417 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:56:53.430999 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:56:53.430943 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" podStartSLOduration=1.430931121 podStartE2EDuration="1.430931121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:56:53.43009864 +0000 UTC m=+1621.895233058" watchObservedRunningTime="2026-07-24 01:56:53.430931121 +0000 UTC m=+1621.896065539" Jul 24 01:57:04.421379 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:04.421346 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:57:27.631731 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.631695 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx"] Jul 24 01:57:27.632219 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.631947 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" podUID="a02fc955-95ba-4773-b629-92390c08de2d" containerName="payload-processing" containerID="cri-o://c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d" gracePeriod=30 Jul 24 01:57:27.872326 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.872301 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:57:27.944759 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.944685 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a02fc955-95ba-4773-b629-92390c08de2d-plugins-config-volume\") pod \"a02fc955-95ba-4773-b629-92390c08de2d\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " Jul 24 01:57:27.944759 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.944741 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhtln\" (UniqueName: \"kubernetes.io/projected/a02fc955-95ba-4773-b629-92390c08de2d-kube-api-access-nhtln\") pod \"a02fc955-95ba-4773-b629-92390c08de2d\" (UID: \"a02fc955-95ba-4773-b629-92390c08de2d\") " Jul 24 01:57:27.945046 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.945023 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a02fc955-95ba-4773-b629-92390c08de2d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a02fc955-95ba-4773-b629-92390c08de2d" (UID: "a02fc955-95ba-4773-b629-92390c08de2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:57:27.946833 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:27.946804 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a02fc955-95ba-4773-b629-92390c08de2d-kube-api-access-nhtln" (OuterVolumeSpecName: "kube-api-access-nhtln") pod "a02fc955-95ba-4773-b629-92390c08de2d" (UID: "a02fc955-95ba-4773-b629-92390c08de2d"). InnerVolumeSpecName "kube-api-access-nhtln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:57:28.045465 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.045432 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a02fc955-95ba-4773-b629-92390c08de2d-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:57:28.045465 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.045461 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhtln\" (UniqueName: \"kubernetes.io/projected/a02fc955-95ba-4773-b629-92390c08de2d-kube-api-access-nhtln\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:57:28.557786 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.557747 2564 generic.go:356] "Generic (PLEG): container finished" podID="a02fc955-95ba-4773-b629-92390c08de2d" containerID="c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d" exitCode=0 Jul 24 01:57:28.557953 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.557798 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" Jul 24 01:57:28.557953 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.557809 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" event={"ID":"a02fc955-95ba-4773-b629-92390c08de2d","Type":"ContainerDied","Data":"c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d"} Jul 24 01:57:28.557953 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.557860 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx" event={"ID":"a02fc955-95ba-4773-b629-92390c08de2d","Type":"ContainerDied","Data":"cbad3ad05915fa382a6565d4578c57b3aa382b8b0a52c8dc810a901939785ca6"} Jul 24 01:57:28.557953 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.557890 2564 scope.go:117] "RemoveContainer" containerID="c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d" Jul 24 01:57:28.565895 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.565875 2564 scope.go:117] "RemoveContainer" containerID="c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d" Jul 24 01:57:28.566175 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:57:28.566150 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d\": container with ID starting with c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d not found: ID does not exist" containerID="c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d" Jul 24 01:57:28.566237 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.566185 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d"} err="failed to get container status \"c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d\": rpc error: code = NotFound desc = could not find container \"c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d\": container with ID starting with c35c84ab84a0974cfe0f309519a835852937ca8d7707e674ff8cd5a621f7a19d not found: ID does not exist" Jul 24 01:57:28.572538 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.572515 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx"] Jul 24 01:57:28.575017 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:28.574995 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-44fb0de5-865745c984-pkmzx"] Jul 24 01:57:29.997409 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:57:29.993226 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a02fc955-95ba-4773-b629-92390c08de2d" path="/var/lib/kubelet/pods/a02fc955-95ba-4773-b629-92390c08de2d/volumes" Jul 24 01:58:20.736658 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.736575 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6"] Jul 24 01:58:20.737035 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.736912 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a02fc955-95ba-4773-b629-92390c08de2d" containerName="payload-processing" Jul 24 01:58:20.737035 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.736925 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a02fc955-95ba-4773-b629-92390c08de2d" containerName="payload-processing" Jul 24 01:58:20.737035 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.736981 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a02fc955-95ba-4773-b629-92390c08de2d" containerName="payload-processing" Jul 24 01:58:20.739860 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.739844 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.742834 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.742796 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-da5bdfda\"" Jul 24 01:58:20.743000 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.742986 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-da5bdfda-dockercfg-mccgz\"" Jul 24 01:58:20.750251 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.750228 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6"] Jul 24 01:58:20.774371 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.768552 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc"] Jul 24 01:58:20.774371 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.772994 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:20.785262 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.785241 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc"] Jul 24 01:58:20.787252 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.787055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-plugins-config-volume\") pod \"payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:20.787252 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.787119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxl54\" (UniqueName: \"kubernetes.io/projected/0d789e77-bc2b-4ad3-871a-b7531305a748-kube-api-access-bxl54\") pod \"payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.787252 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.787217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kndw4\" (UniqueName: \"kubernetes.io/projected/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-kube-api-access-kndw4\") pod \"payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:20.787454 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.787268 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d789e77-bc2b-4ad3-871a-b7531305a748-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.887657 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.887623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kndw4\" (UniqueName: \"kubernetes.io/projected/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-kube-api-access-kndw4\") pod \"payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:20.887826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.887675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d789e77-bc2b-4ad3-871a-b7531305a748-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.887826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.887729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-plugins-config-volume\") pod \"payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:20.887826 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.887766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxl54\" (UniqueName: \"kubernetes.io/projected/0d789e77-bc2b-4ad3-871a-b7531305a748-kube-api-access-bxl54\") pod \"payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.888515 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.888489 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d789e77-bc2b-4ad3-871a-b7531305a748-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.888615 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.888523 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-plugins-config-volume\") pod \"payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:20.896023 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.896000 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxl54\" (UniqueName: \"kubernetes.io/projected/0d789e77-bc2b-4ad3-871a-b7531305a748-kube-api-access-bxl54\") pod \"payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:20.896103 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:20.896081 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kndw4\" (UniqueName: \"kubernetes.io/projected/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-kube-api-access-kndw4\") pod \"payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:21.051328 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.051236 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:21.084016 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.083988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:21.195564 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.195372 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6"] Jul 24 01:58:21.197740 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:58:21.197709 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d789e77_bc2b_4ad3_871a_b7531305a748.slice/crio-40adad783d188e0acc7c3cb18e39b5ddddab7f11d510da8eb34b342d1ac515d2 WatchSource:0}: Error finding container 40adad783d188e0acc7c3cb18e39b5ddddab7f11d510da8eb34b342d1ac515d2: Status 404 returned error can't find the container with id 40adad783d188e0acc7c3cb18e39b5ddddab7f11d510da8eb34b342d1ac515d2 Jul 24 01:58:21.221714 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.221688 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc"] Jul 24 01:58:21.235273 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:58:21.235246 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f5e59a_fa89_40b6_9fc2_f515f8f6ee0e.slice/crio-ff003737250e693376372bb6a99bfa01acfa56492d860967a284f3841a7bf70e WatchSource:0}: Error finding container ff003737250e693376372bb6a99bfa01acfa56492d860967a284f3841a7bf70e: Status 404 returned error can't find the container with id ff003737250e693376372bb6a99bfa01acfa56492d860967a284f3841a7bf70e Jul 24 01:58:21.775272 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.775232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" event={"ID":"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e","Type":"ContainerStarted","Data":"501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2"} Jul 24 01:58:21.775967 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.775295 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" event={"ID":"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e","Type":"ContainerStarted","Data":"ff003737250e693376372bb6a99bfa01acfa56492d860967a284f3841a7bf70e"} Jul 24 01:58:21.775967 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.775391 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:21.776938 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.776906 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" event={"ID":"0d789e77-bc2b-4ad3-871a-b7531305a748","Type":"ContainerStarted","Data":"c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c"} Jul 24 01:58:21.777069 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.776956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" event={"ID":"0d789e77-bc2b-4ad3-871a-b7531305a748","Type":"ContainerStarted","Data":"40adad783d188e0acc7c3cb18e39b5ddddab7f11d510da8eb34b342d1ac515d2"} Jul 24 01:58:21.777069 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.777011 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:21.791401 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.791350 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" podStartSLOduration=1.7913354529999999 podStartE2EDuration="1.791335453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:58:21.790575568 +0000 UTC m=+1710.255710009" watchObservedRunningTime="2026-07-24 01:58:21.791335453 +0000 UTC m=+1710.256469870" Jul 24 01:58:21.805862 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:21.805816 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" podStartSLOduration=1.805801395 podStartE2EDuration="1.805801395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:58:21.80445555 +0000 UTC m=+1710.269589968" watchObservedRunningTime="2026-07-24 01:58:21.805801395 +0000 UTC m=+1710.270935811" Jul 24 01:58:32.783584 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:32.783549 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:32.784037 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:32.783624 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:56.265343 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.265309 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc"] Jul 24 01:58:56.265800 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.265562 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" podUID="e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" containerName="payload-processing" containerID="cri-o://501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2" gracePeriod=30 Jul 24 01:58:56.276858 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.276826 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6"] Jul 24 01:58:56.277513 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.277059 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" podUID="0d789e77-bc2b-4ad3-871a-b7531305a748" containerName="payload-pre-processing" containerID="cri-o://c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c" gracePeriod=30 Jul 24 01:58:56.536501 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.536470 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:56.558231 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.558202 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:56.687708 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.687659 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d789e77-bc2b-4ad3-871a-b7531305a748-plugins-config-volume\") pod \"0d789e77-bc2b-4ad3-871a-b7531305a748\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " Jul 24 01:58:56.687708 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.687711 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-plugins-config-volume\") pod \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " Jul 24 01:58:56.687921 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.687742 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kndw4\" (UniqueName: \"kubernetes.io/projected/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-kube-api-access-kndw4\") pod \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\" (UID: \"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e\") " Jul 24 01:58:56.687921 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.687767 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxl54\" (UniqueName: \"kubernetes.io/projected/0d789e77-bc2b-4ad3-871a-b7531305a748-kube-api-access-bxl54\") pod \"0d789e77-bc2b-4ad3-871a-b7531305a748\" (UID: \"0d789e77-bc2b-4ad3-871a-b7531305a748\") " Jul 24 01:58:56.688178 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.688157 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" (UID: "e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:58:56.688255 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.688157 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d789e77-bc2b-4ad3-871a-b7531305a748-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0d789e77-bc2b-4ad3-871a-b7531305a748" (UID: "0d789e77-bc2b-4ad3-871a-b7531305a748"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:58:56.689903 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.689868 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d789e77-bc2b-4ad3-871a-b7531305a748-kube-api-access-bxl54" (OuterVolumeSpecName: "kube-api-access-bxl54") pod "0d789e77-bc2b-4ad3-871a-b7531305a748" (UID: "0d789e77-bc2b-4ad3-871a-b7531305a748"). InnerVolumeSpecName "kube-api-access-bxl54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:58:56.689959 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.689897 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-kube-api-access-kndw4" (OuterVolumeSpecName: "kube-api-access-kndw4") pod "e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" (UID: "e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e"). InnerVolumeSpecName "kube-api-access-kndw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:58:56.789352 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.789276 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d789e77-bc2b-4ad3-871a-b7531305a748-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:58:56.789352 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.789304 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:58:56.789352 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.789316 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kndw4\" (UniqueName: \"kubernetes.io/projected/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e-kube-api-access-kndw4\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:58:56.789352 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.789325 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxl54\" (UniqueName: \"kubernetes.io/projected/0d789e77-bc2b-4ad3-871a-b7531305a748-kube-api-access-bxl54\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:58:56.923318 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.923277 2564 generic.go:356] "Generic (PLEG): container finished" podID="0d789e77-bc2b-4ad3-871a-b7531305a748" containerID="c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c" exitCode=0 Jul 24 01:58:56.923504 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.923326 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" Jul 24 01:58:56.923504 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.923318 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" event={"ID":"0d789e77-bc2b-4ad3-871a-b7531305a748","Type":"ContainerDied","Data":"c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c"} Jul 24 01:58:56.923504 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.923497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6" event={"ID":"0d789e77-bc2b-4ad3-871a-b7531305a748","Type":"ContainerDied","Data":"40adad783d188e0acc7c3cb18e39b5ddddab7f11d510da8eb34b342d1ac515d2"} Jul 24 01:58:56.923671 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.923534 2564 scope.go:117] "RemoveContainer" containerID="c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c" Jul 24 01:58:56.924687 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.924669 2564 generic.go:356] "Generic (PLEG): container finished" podID="e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" containerID="501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2" exitCode=0 Jul 24 01:58:56.924792 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.924697 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" event={"ID":"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e","Type":"ContainerDied","Data":"501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2"} Jul 24 01:58:56.924792 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.924715 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" Jul 24 01:58:56.924792 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.924730 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc" event={"ID":"e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e","Type":"ContainerDied","Data":"ff003737250e693376372bb6a99bfa01acfa56492d860967a284f3841a7bf70e"} Jul 24 01:58:56.932977 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.932961 2564 scope.go:117] "RemoveContainer" containerID="c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c" Jul 24 01:58:56.933242 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:58:56.933217 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c\": container with ID starting with c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c not found: ID does not exist" containerID="c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c" Jul 24 01:58:56.933291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.933253 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c"} err="failed to get container status \"c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c\": rpc error: code = NotFound desc = could not find container \"c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c\": container with ID starting with c3f32e7665d6d37f0c925837b6b6d552434bdfbe18fbadd664be274e5d081d3c not found: ID does not exist" Jul 24 01:58:56.933291 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.933271 2564 scope.go:117] "RemoveContainer" containerID="501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2" Jul 24 01:58:56.940646 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.940630 2564 scope.go:117] "RemoveContainer" containerID="501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2" Jul 24 01:58:56.940870 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:58:56.940851 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2\": container with ID starting with 501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2 not found: ID does not exist" containerID="501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2" Jul 24 01:58:56.940913 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.940877 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2"} err="failed to get container status \"501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2\": rpc error: code = NotFound desc = could not find container \"501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2\": container with ID starting with 501bc7999a0a293cd9fb5a09660340c2c55238c6e15f52bc6b91e5f593dc75d2 not found: ID does not exist" Jul 24 01:58:56.944378 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.944358 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6"] Jul 24 01:58:56.948449 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.948418 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-da5bdfda-7bfdc95ffb-7lgq6"] Jul 24 01:58:56.959835 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.959810 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc"] Jul 24 01:58:56.963694 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:56.963674 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-da5bdfda-f87c8dccd-q4ctc"] Jul 24 01:58:57.992054 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:57.992022 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d789e77-bc2b-4ad3-871a-b7531305a748" path="/var/lib/kubelet/pods/0d789e77-bc2b-4ad3-871a-b7531305a748/volumes" Jul 24 01:58:57.992429 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:58:57.992381 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" path="/var/lib/kubelet/pods/e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e/volumes" Jul 24 01:59:06.009308 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009272 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25"] Jul 24 01:59:06.009770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009577 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" containerName="payload-processing" Jul 24 01:59:06.009770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009589 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" containerName="payload-processing" Jul 24 01:59:06.009770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009604 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d789e77-bc2b-4ad3-871a-b7531305a748" containerName="payload-pre-processing" Jul 24 01:59:06.009770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009609 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d789e77-bc2b-4ad3-871a-b7531305a748" containerName="payload-pre-processing" Jul 24 01:59:06.009770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009679 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d789e77-bc2b-4ad3-871a-b7531305a748" containerName="payload-pre-processing" Jul 24 01:59:06.009770 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.009689 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5f5e59a-fa89-40b6-9fc2-f515f8f6ee0e" containerName="payload-processing" Jul 24 01:59:06.014015 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.013998 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.017062 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.017040 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-d1346920\"" Jul 24 01:59:06.017195 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.017100 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-d1346920-dockercfg-46wjw\"" Jul 24 01:59:06.025092 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.025069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25"] Jul 24 01:59:06.037805 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.037781 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78"] Jul 24 01:59:06.040822 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.040804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.051483 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.051456 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78"] Jul 24 01:59:06.166774 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.166738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e08e3fe-f146-4eb0-b01a-43bf60135b37-plugins-config-volume\") pod \"payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.166774 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.166780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcq25\" (UniqueName: \"kubernetes.io/projected/3e08e3fe-f146-4eb0-b01a-43bf60135b37-kube-api-access-lcq25\") pod \"payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.167026 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.166808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5hf6\" (UniqueName: \"kubernetes.io/projected/b251ed92-15cd-40d0-9aa3-aa153235056e-kube-api-access-n5hf6\") pod \"payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.167026 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.166912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b251ed92-15cd-40d0-9aa3-aa153235056e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.268011 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.267928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e08e3fe-f146-4eb0-b01a-43bf60135b37-plugins-config-volume\") pod \"payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.268011 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.267970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcq25\" (UniqueName: \"kubernetes.io/projected/3e08e3fe-f146-4eb0-b01a-43bf60135b37-kube-api-access-lcq25\") pod \"payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.268011 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.267991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5hf6\" (UniqueName: \"kubernetes.io/projected/b251ed92-15cd-40d0-9aa3-aa153235056e-kube-api-access-n5hf6\") pod \"payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.268257 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.268015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b251ed92-15cd-40d0-9aa3-aa153235056e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.268707 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.268688 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b251ed92-15cd-40d0-9aa3-aa153235056e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.268742 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.268714 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e08e3fe-f146-4eb0-b01a-43bf60135b37-plugins-config-volume\") pod \"payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.284791 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.284757 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcq25\" (UniqueName: \"kubernetes.io/projected/3e08e3fe-f146-4eb0-b01a-43bf60135b37-kube-api-access-lcq25\") pod \"payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.285097 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.285075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5hf6\" (UniqueName: \"kubernetes.io/projected/b251ed92-15cd-40d0-9aa3-aa153235056e-kube-api-access-n5hf6\") pod \"payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.323226 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.323193 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.356790 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.356758 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.465867 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.465816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25"] Jul 24 01:59:06.471044 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:59:06.471015 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb251ed92_15cd_40d0_9aa3_aa153235056e.slice/crio-f540265d95384c1bfc7ea4c291bdce8d893008f1c91dab45ba8b1f0f0b3a8d36 WatchSource:0}: Error finding container f540265d95384c1bfc7ea4c291bdce8d893008f1c91dab45ba8b1f0f0b3a8d36: Status 404 returned error can't find the container with id f540265d95384c1bfc7ea4c291bdce8d893008f1c91dab45ba8b1f0f0b3a8d36 Jul 24 01:59:06.495363 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.495335 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78"] Jul 24 01:59:06.498361 ip-10-0-138-219 kubenswrapper[2564]: W0724 01:59:06.498331 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e08e3fe_f146_4eb0_b01a_43bf60135b37.slice/crio-294e6e8aac5ace3c4fb50d2561daa304e87db849d074b331cca4d5352a3ec272 WatchSource:0}: Error finding container 294e6e8aac5ace3c4fb50d2561daa304e87db849d074b331cca4d5352a3ec272: Status 404 returned error can't find the container with id 294e6e8aac5ace3c4fb50d2561daa304e87db849d074b331cca4d5352a3ec272 Jul 24 01:59:06.979704 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.979668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" event={"ID":"3e08e3fe-f146-4eb0-b01a-43bf60135b37","Type":"ContainerStarted","Data":"7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28"} Jul 24 01:59:06.979874 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.979732 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" event={"ID":"3e08e3fe-f146-4eb0-b01a-43bf60135b37","Type":"ContainerStarted","Data":"294e6e8aac5ace3c4fb50d2561daa304e87db849d074b331cca4d5352a3ec272"} Jul 24 01:59:06.979874 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.979869 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:06.981162 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.981115 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" event={"ID":"b251ed92-15cd-40d0-9aa3-aa153235056e","Type":"ContainerStarted","Data":"1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08"} Jul 24 01:59:06.981282 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.981184 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" event={"ID":"b251ed92-15cd-40d0-9aa3-aa153235056e","Type":"ContainerStarted","Data":"f540265d95384c1bfc7ea4c291bdce8d893008f1c91dab45ba8b1f0f0b3a8d36"} Jul 24 01:59:06.981282 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.981213 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:06.996272 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:06.996228 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" podStartSLOduration=0.996211763 podStartE2EDuration="996.211763ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:59:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:59:06.995030754 +0000 UTC m=+1755.460165172" watchObservedRunningTime="2026-07-24 01:59:06.996211763 +0000 UTC m=+1755.461346180" Jul 24 01:59:07.009715 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:07.009670 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" podStartSLOduration=2.009654798 podStartE2EDuration="2.009654798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 01:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 01:59:07.009150983 +0000 UTC m=+1755.474285413" watchObservedRunningTime="2026-07-24 01:59:07.009654798 +0000 UTC m=+1755.474789247" Jul 24 01:59:17.992138 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:17.992095 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:17.992542 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:17.992167 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:40.743466 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:40.743412 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78"] Jul 24 01:59:40.744005 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:40.743676 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" podUID="3e08e3fe-f146-4eb0-b01a-43bf60135b37" containerName="payload-processing" containerID="cri-o://7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28" gracePeriod=30 Jul 24 01:59:40.749741 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:40.749706 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25"] Jul 24 01:59:40.749941 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:40.749915 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" podUID="b251ed92-15cd-40d0-9aa3-aa153235056e" containerName="payload-pre-processing" containerID="cri-o://1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08" gracePeriod=30 Jul 24 01:59:41.002901 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.002879 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:41.005621 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.005602 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:41.124427 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.124389 2564 generic.go:356] "Generic (PLEG): container finished" podID="b251ed92-15cd-40d0-9aa3-aa153235056e" containerID="1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08" exitCode=0 Jul 24 01:59:41.124613 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.124434 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" Jul 24 01:59:41.124613 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.124475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" event={"ID":"b251ed92-15cd-40d0-9aa3-aa153235056e","Type":"ContainerDied","Data":"1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08"} Jul 24 01:59:41.124613 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.124524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25" event={"ID":"b251ed92-15cd-40d0-9aa3-aa153235056e","Type":"ContainerDied","Data":"f540265d95384c1bfc7ea4c291bdce8d893008f1c91dab45ba8b1f0f0b3a8d36"} Jul 24 01:59:41.124613 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.124547 2564 scope.go:117] "RemoveContainer" containerID="1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08" Jul 24 01:59:41.125687 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.125664 2564 generic.go:356] "Generic (PLEG): container finished" podID="3e08e3fe-f146-4eb0-b01a-43bf60135b37" containerID="7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28" exitCode=0 Jul 24 01:59:41.125772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.125704 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" Jul 24 01:59:41.125772 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.125744 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" event={"ID":"3e08e3fe-f146-4eb0-b01a-43bf60135b37","Type":"ContainerDied","Data":"7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28"} Jul 24 01:59:41.125841 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.125781 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78" event={"ID":"3e08e3fe-f146-4eb0-b01a-43bf60135b37","Type":"ContainerDied","Data":"294e6e8aac5ace3c4fb50d2561daa304e87db849d074b331cca4d5352a3ec272"} Jul 24 01:59:41.132528 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.132513 2564 scope.go:117] "RemoveContainer" containerID="1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08" Jul 24 01:59:41.132743 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:59:41.132726 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08\": container with ID starting with 1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08 not found: ID does not exist" containerID="1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08" Jul 24 01:59:41.132784 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.132751 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08"} err="failed to get container status \"1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08\": rpc error: code = NotFound desc = could not find container \"1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08\": container with ID starting with 1ed240e3cacc6fc00a2f03160b6f2c399b870929c2ac9e768c9cefe8f2270c08 not found: ID does not exist" Jul 24 01:59:41.132784 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.132767 2564 scope.go:117] "RemoveContainer" containerID="7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28" Jul 24 01:59:41.139737 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.139722 2564 scope.go:117] "RemoveContainer" containerID="7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28" Jul 24 01:59:41.139963 ip-10-0-138-219 kubenswrapper[2564]: E0724 01:59:41.139946 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28\": container with ID starting with 7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28 not found: ID does not exist" containerID="7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28" Jul 24 01:59:41.140007 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.139969 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28"} err="failed to get container status \"7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28\": rpc error: code = NotFound desc = could not find container \"7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28\": container with ID starting with 7d10efc249b7dcf352cbdab88e8db9a109d4da3a4d627b6dd171062849e75d28 not found: ID does not exist" Jul 24 01:59:41.176273 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.176250 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b251ed92-15cd-40d0-9aa3-aa153235056e-plugins-config-volume\") pod \"b251ed92-15cd-40d0-9aa3-aa153235056e\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " Jul 24 01:59:41.176351 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.176277 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e08e3fe-f146-4eb0-b01a-43bf60135b37-plugins-config-volume\") pod \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " Jul 24 01:59:41.176351 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.176345 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5hf6\" (UniqueName: \"kubernetes.io/projected/b251ed92-15cd-40d0-9aa3-aa153235056e-kube-api-access-n5hf6\") pod \"b251ed92-15cd-40d0-9aa3-aa153235056e\" (UID: \"b251ed92-15cd-40d0-9aa3-aa153235056e\") " Jul 24 01:59:41.176440 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.176363 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcq25\" (UniqueName: \"kubernetes.io/projected/3e08e3fe-f146-4eb0-b01a-43bf60135b37-kube-api-access-lcq25\") pod \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\" (UID: \"3e08e3fe-f146-4eb0-b01a-43bf60135b37\") " Jul 24 01:59:41.176644 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.176620 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b251ed92-15cd-40d0-9aa3-aa153235056e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b251ed92-15cd-40d0-9aa3-aa153235056e" (UID: "b251ed92-15cd-40d0-9aa3-aa153235056e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:59:41.176721 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.176698 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e08e3fe-f146-4eb0-b01a-43bf60135b37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e08e3fe-f146-4eb0-b01a-43bf60135b37" (UID: "3e08e3fe-f146-4eb0-b01a-43bf60135b37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 01:59:41.178199 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.178173 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b251ed92-15cd-40d0-9aa3-aa153235056e-kube-api-access-n5hf6" (OuterVolumeSpecName: "kube-api-access-n5hf6") pod "b251ed92-15cd-40d0-9aa3-aa153235056e" (UID: "b251ed92-15cd-40d0-9aa3-aa153235056e"). InnerVolumeSpecName "kube-api-access-n5hf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:59:41.178307 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.178287 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e08e3fe-f146-4eb0-b01a-43bf60135b37-kube-api-access-lcq25" (OuterVolumeSpecName: "kube-api-access-lcq25") pod "3e08e3fe-f146-4eb0-b01a-43bf60135b37" (UID: "3e08e3fe-f146-4eb0-b01a-43bf60135b37"). InnerVolumeSpecName "kube-api-access-lcq25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 01:59:41.277023 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.276938 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5hf6\" (UniqueName: \"kubernetes.io/projected/b251ed92-15cd-40d0-9aa3-aa153235056e-kube-api-access-n5hf6\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:59:41.277023 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.276969 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcq25\" (UniqueName: \"kubernetes.io/projected/3e08e3fe-f146-4eb0-b01a-43bf60135b37-kube-api-access-lcq25\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:59:41.277023 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.276978 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b251ed92-15cd-40d0-9aa3-aa153235056e-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:59:41.277023 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.276986 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e08e3fe-f146-4eb0-b01a-43bf60135b37-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 01:59:41.448471 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.448440 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25"] Jul 24 01:59:41.452238 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.452216 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d1346920-b48b9c45b-5lr25"] Jul 24 01:59:41.462419 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.462396 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78"] Jul 24 01:59:41.465713 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.465691 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d1346920-7c8f785d74-mhb78"] Jul 24 01:59:41.993186 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.993155 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e08e3fe-f146-4eb0-b01a-43bf60135b37" path="/var/lib/kubelet/pods/3e08e3fe-f146-4eb0-b01a-43bf60135b37/volumes" Jul 24 01:59:41.993546 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:41.993493 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b251ed92-15cd-40d0-9aa3-aa153235056e" path="/var/lib/kubelet/pods/b251ed92-15cd-40d0-9aa3-aa153235056e/volumes" Jul 24 01:59:51.933659 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:51.933632 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 01:59:51.940274 ip-10-0-138-219 kubenswrapper[2564]: I0724 01:59:51.940252 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:01:36.164810 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.164733 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf"] Jul 24 02:01:36.165269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.165051 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e08e3fe-f146-4eb0-b01a-43bf60135b37" containerName="payload-processing" Jul 24 02:01:36.165269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.165064 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e08e3fe-f146-4eb0-b01a-43bf60135b37" containerName="payload-processing" Jul 24 02:01:36.165269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.165082 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b251ed92-15cd-40d0-9aa3-aa153235056e" containerName="payload-pre-processing" Jul 24 02:01:36.165269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.165089 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b251ed92-15cd-40d0-9aa3-aa153235056e" containerName="payload-pre-processing" Jul 24 02:01:36.165269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.165174 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e08e3fe-f146-4eb0-b01a-43bf60135b37" containerName="payload-processing" Jul 24 02:01:36.165269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.165186 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b251ed92-15cd-40d0-9aa3-aa153235056e" containerName="payload-pre-processing" Jul 24 02:01:36.168034 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.168016 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.170913 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.170893 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-20b5fb51-dockercfg-6rzfs\"" Jul 24 02:01:36.171004 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.170893 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-20b5fb51\"" Jul 24 02:01:36.181637 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.181617 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf"] Jul 24 02:01:36.347976 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.347935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vzkx\" (UniqueName: \"kubernetes.io/projected/7bfff9d6-4498-426f-b18f-9fa09d161e2b-kube-api-access-5vzkx\") pod \"payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.348175 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.347997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfff9d6-4498-426f-b18f-9fa09d161e2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.448637 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.448542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzkx\" (UniqueName: \"kubernetes.io/projected/7bfff9d6-4498-426f-b18f-9fa09d161e2b-kube-api-access-5vzkx\") pod \"payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.448637 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.448600 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfff9d6-4498-426f-b18f-9fa09d161e2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.449287 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.449269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfff9d6-4498-426f-b18f-9fa09d161e2b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.458332 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.458307 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vzkx\" (UniqueName: \"kubernetes.io/projected/7bfff9d6-4498-426f-b18f-9fa09d161e2b-kube-api-access-5vzkx\") pod \"payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.478379 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.478354 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:36.606856 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:36.606821 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf"] Jul 24 02:01:36.609510 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:01:36.609483 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfff9d6_4498_426f_b18f_9fa09d161e2b.slice/crio-a735f8fe00293569f1d3b2f9d7c5ada37283a3708ea9dc78ec53bd4a5a4011c9 WatchSource:0}: Error finding container a735f8fe00293569f1d3b2f9d7c5ada37283a3708ea9dc78ec53bd4a5a4011c9: Status 404 returned error can't find the container with id a735f8fe00293569f1d3b2f9d7c5ada37283a3708ea9dc78ec53bd4a5a4011c9 Jul 24 02:01:37.616804 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:37.616769 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" event={"ID":"7bfff9d6-4498-426f-b18f-9fa09d161e2b","Type":"ContainerStarted","Data":"bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd"} Jul 24 02:01:37.617218 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:37.616822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" event={"ID":"7bfff9d6-4498-426f-b18f-9fa09d161e2b","Type":"ContainerStarted","Data":"a735f8fe00293569f1d3b2f9d7c5ada37283a3708ea9dc78ec53bd4a5a4011c9"} Jul 24 02:01:37.617218 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:37.616851 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:37.638724 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:37.638673 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" podStartSLOduration=1.638653321 podStartE2EDuration="1.638653321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:01:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:01:37.637956872 +0000 UTC m=+1906.103091289" watchObservedRunningTime="2026-07-24 02:01:37.638653321 +0000 UTC m=+1906.103787741" Jul 24 02:01:48.623299 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:48.623267 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:59.456993 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.456958 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf"] Jul 24 02:01:59.459607 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.457229 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" podUID="7bfff9d6-4498-426f-b18f-9fa09d161e2b" containerName="payload-pre-processing" containerID="cri-o://bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd" gracePeriod=30 Jul 24 02:01:59.699335 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.699312 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:59.707560 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.707492 2564 generic.go:356] "Generic (PLEG): container finished" podID="7bfff9d6-4498-426f-b18f-9fa09d161e2b" containerID="bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd" exitCode=0 Jul 24 02:01:59.707560 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.707529 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" Jul 24 02:01:59.707689 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.707570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" event={"ID":"7bfff9d6-4498-426f-b18f-9fa09d161e2b","Type":"ContainerDied","Data":"bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd"} Jul 24 02:01:59.707689 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.707622 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf" event={"ID":"7bfff9d6-4498-426f-b18f-9fa09d161e2b","Type":"ContainerDied","Data":"a735f8fe00293569f1d3b2f9d7c5ada37283a3708ea9dc78ec53bd4a5a4011c9"} Jul 24 02:01:59.707689 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.707646 2564 scope.go:117] "RemoveContainer" containerID="bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd" Jul 24 02:01:59.715067 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.715051 2564 scope.go:117] "RemoveContainer" containerID="bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd" Jul 24 02:01:59.715415 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:01:59.715394 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd\": container with ID starting with bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd not found: ID does not exist" containerID="bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd" Jul 24 02:01:59.715487 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.715425 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd"} err="failed to get container status \"bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd\": rpc error: code = NotFound desc = could not find container \"bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd\": container with ID starting with bff8defe2914f9df654b2284c315b676686035c8d536a65f8a180690156cd7fd not found: ID does not exist" Jul 24 02:01:59.740434 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.740415 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfff9d6-4498-426f-b18f-9fa09d161e2b-plugins-config-volume\") pod \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " Jul 24 02:01:59.740547 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.740457 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vzkx\" (UniqueName: \"kubernetes.io/projected/7bfff9d6-4498-426f-b18f-9fa09d161e2b-kube-api-access-5vzkx\") pod \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\" (UID: \"7bfff9d6-4498-426f-b18f-9fa09d161e2b\") " Jul 24 02:01:59.740803 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.740770 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bfff9d6-4498-426f-b18f-9fa09d161e2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7bfff9d6-4498-426f-b18f-9fa09d161e2b" (UID: "7bfff9d6-4498-426f-b18f-9fa09d161e2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:01:59.742783 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.742760 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfff9d6-4498-426f-b18f-9fa09d161e2b-kube-api-access-5vzkx" (OuterVolumeSpecName: "kube-api-access-5vzkx") pod "7bfff9d6-4498-426f-b18f-9fa09d161e2b" (UID: "7bfff9d6-4498-426f-b18f-9fa09d161e2b"). InnerVolumeSpecName "kube-api-access-5vzkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:01:59.841289 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.841258 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bfff9d6-4498-426f-b18f-9fa09d161e2b-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:01:59.841289 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:01:59.841283 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vzkx\" (UniqueName: \"kubernetes.io/projected/7bfff9d6-4498-426f-b18f-9fa09d161e2b-kube-api-access-5vzkx\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:02:00.031735 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:00.031700 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf"] Jul 24 02:02:00.040174 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:00.040144 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-20b5fb51-77775d4ddd-rxvwf"] Jul 24 02:02:01.991724 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:01.991689 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bfff9d6-4498-426f-b18f-9fa09d161e2b" path="/var/lib/kubelet/pods/7bfff9d6-4498-426f-b18f-9fa09d161e2b/volumes" Jul 24 02:02:38.786306 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.786271 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m"] Jul 24 02:02:38.786910 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.786702 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bfff9d6-4498-426f-b18f-9fa09d161e2b" containerName="payload-pre-processing" Jul 24 02:02:38.786910 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.786721 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bfff9d6-4498-426f-b18f-9fa09d161e2b" containerName="payload-pre-processing" Jul 24 02:02:38.786910 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.786797 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bfff9d6-4498-426f-b18f-9fa09d161e2b" containerName="payload-pre-processing" Jul 24 02:02:38.789738 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.789716 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:38.796588 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.796571 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f"] Jul 24 02:02:38.799532 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.799517 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:38.800262 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.800242 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d043f7ca-dockercfg-rhhjc\"" Jul 24 02:02:38.801075 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.801057 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d043f7ca\"" Jul 24 02:02:38.828710 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.828684 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m"] Jul 24 02:02:38.855608 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.855585 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f"] Jul 24 02:02:38.867766 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.867741 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr9qj\" (UniqueName: \"kubernetes.io/projected/536038ed-210f-4f88-a777-82693acbdf6a-kube-api-access-zr9qj\") pod \"payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:38.867888 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.867787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/536038ed-210f-4f88-a777-82693acbdf6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:38.968369 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.968337 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/536038ed-210f-4f88-a777-82693acbdf6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:38.968533 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.968392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh885\" (UniqueName: \"kubernetes.io/projected/8059c2bc-282f-4bf5-8240-81218595b6e4-kube-api-access-xh885\") pod \"payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:38.968533 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.968435 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr9qj\" (UniqueName: \"kubernetes.io/projected/536038ed-210f-4f88-a777-82693acbdf6a-kube-api-access-zr9qj\") pod \"payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:38.968533 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.968458 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8059c2bc-282f-4bf5-8240-81218595b6e4-plugins-config-volume\") pod \"payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:38.969011 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.968991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/536038ed-210f-4f88-a777-82693acbdf6a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:38.978880 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:38.978853 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr9qj\" (UniqueName: \"kubernetes.io/projected/536038ed-210f-4f88-a777-82693acbdf6a-kube-api-access-zr9qj\") pod \"payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:39.069423 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.069339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh885\" (UniqueName: \"kubernetes.io/projected/8059c2bc-282f-4bf5-8240-81218595b6e4-kube-api-access-xh885\") pod \"payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:39.069564 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.069427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8059c2bc-282f-4bf5-8240-81218595b6e4-plugins-config-volume\") pod \"payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:39.070104 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.070084 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8059c2bc-282f-4bf5-8240-81218595b6e4-plugins-config-volume\") pod \"payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:39.086320 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.086292 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh885\" (UniqueName: \"kubernetes.io/projected/8059c2bc-282f-4bf5-8240-81218595b6e4-kube-api-access-xh885\") pod \"payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:39.100152 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.100112 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:39.108269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.108250 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:39.252818 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.252738 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m"] Jul 24 02:02:39.255049 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:02:39.255023 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536038ed_210f_4f88_a777_82693acbdf6a.slice/crio-ea504aebc97f4b186391aa1bf539ffd95df1bdb1757ab9c50751ee57d6837154 WatchSource:0}: Error finding container ea504aebc97f4b186391aa1bf539ffd95df1bdb1757ab9c50751ee57d6837154: Status 404 returned error can't find the container with id ea504aebc97f4b186391aa1bf539ffd95df1bdb1757ab9c50751ee57d6837154 Jul 24 02:02:39.293647 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.293624 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f"] Jul 24 02:02:39.295920 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:02:39.295893 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8059c2bc_282f_4bf5_8240_81218595b6e4.slice/crio-1229725ee0239fbb27119cee56afc1a9eaebc6268d771456a48e34bfd3620d44 WatchSource:0}: Error finding container 1229725ee0239fbb27119cee56afc1a9eaebc6268d771456a48e34bfd3620d44: Status 404 returned error can't find the container with id 1229725ee0239fbb27119cee56afc1a9eaebc6268d771456a48e34bfd3620d44 Jul 24 02:02:39.869320 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.869285 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" event={"ID":"8059c2bc-282f-4bf5-8240-81218595b6e4","Type":"ContainerStarted","Data":"d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f"} Jul 24 02:02:39.869804 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.869346 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" event={"ID":"8059c2bc-282f-4bf5-8240-81218595b6e4","Type":"ContainerStarted","Data":"1229725ee0239fbb27119cee56afc1a9eaebc6268d771456a48e34bfd3620d44"} Jul 24 02:02:39.869804 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.869392 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:39.870880 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.870855 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" event={"ID":"536038ed-210f-4f88-a777-82693acbdf6a","Type":"ContainerStarted","Data":"054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691"} Jul 24 02:02:39.870988 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.870894 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" event={"ID":"536038ed-210f-4f88-a777-82693acbdf6a","Type":"ContainerStarted","Data":"ea504aebc97f4b186391aa1bf539ffd95df1bdb1757ab9c50751ee57d6837154"} Jul 24 02:02:39.870988 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.870978 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:39.897806 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:39.897762 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" podStartSLOduration=1.897747351 podStartE2EDuration="1.897747351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:02:39.896617006 +0000 UTC m=+1968.361751452" watchObservedRunningTime="2026-07-24 02:02:39.897747351 +0000 UTC m=+1968.362881769" Jul 24 02:02:50.877987 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:50.877953 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:02:50.878481 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:50.878023 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:02:50.905528 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:02:50.905477 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" podStartSLOduration=12.90546069 podStartE2EDuration="12.90546069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:02:39.933629832 +0000 UTC m=+1968.398764269" watchObservedRunningTime="2026-07-24 02:02:50.90546069 +0000 UTC m=+1979.370595106" Jul 24 02:03:31.978560 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:31.978525 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f"] Jul 24 02:03:31.979188 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:31.978769 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" podUID="8059c2bc-282f-4bf5-8240-81218595b6e4" containerName="payload-processing" containerID="cri-o://d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f" gracePeriod=30 Jul 24 02:03:31.985812 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:31.985785 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m"] Jul 24 02:03:31.986020 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:31.985993 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" podUID="536038ed-210f-4f88-a777-82693acbdf6a" containerName="payload-pre-processing" containerID="cri-o://054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691" gracePeriod=30 Jul 24 02:03:32.251753 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.251725 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:03:32.254461 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.254442 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:03:32.405434 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.405406 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8059c2bc-282f-4bf5-8240-81218595b6e4-plugins-config-volume\") pod \"8059c2bc-282f-4bf5-8240-81218595b6e4\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " Jul 24 02:03:32.405608 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.405441 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh885\" (UniqueName: \"kubernetes.io/projected/8059c2bc-282f-4bf5-8240-81218595b6e4-kube-api-access-xh885\") pod \"8059c2bc-282f-4bf5-8240-81218595b6e4\" (UID: \"8059c2bc-282f-4bf5-8240-81218595b6e4\") " Jul 24 02:03:32.405608 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.405486 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/536038ed-210f-4f88-a777-82693acbdf6a-plugins-config-volume\") pod \"536038ed-210f-4f88-a777-82693acbdf6a\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " Jul 24 02:03:32.405608 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.405528 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zr9qj\" (UniqueName: \"kubernetes.io/projected/536038ed-210f-4f88-a777-82693acbdf6a-kube-api-access-zr9qj\") pod \"536038ed-210f-4f88-a777-82693acbdf6a\" (UID: \"536038ed-210f-4f88-a777-82693acbdf6a\") " Jul 24 02:03:32.405846 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.405824 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536038ed-210f-4f88-a777-82693acbdf6a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "536038ed-210f-4f88-a777-82693acbdf6a" (UID: "536038ed-210f-4f88-a777-82693acbdf6a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:03:32.405919 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.405874 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8059c2bc-282f-4bf5-8240-81218595b6e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8059c2bc-282f-4bf5-8240-81218595b6e4" (UID: "8059c2bc-282f-4bf5-8240-81218595b6e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:03:32.407586 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.407560 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8059c2bc-282f-4bf5-8240-81218595b6e4-kube-api-access-xh885" (OuterVolumeSpecName: "kube-api-access-xh885") pod "8059c2bc-282f-4bf5-8240-81218595b6e4" (UID: "8059c2bc-282f-4bf5-8240-81218595b6e4"). InnerVolumeSpecName "kube-api-access-xh885". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:03:32.407668 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.407614 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/536038ed-210f-4f88-a777-82693acbdf6a-kube-api-access-zr9qj" (OuterVolumeSpecName: "kube-api-access-zr9qj") pod "536038ed-210f-4f88-a777-82693acbdf6a" (UID: "536038ed-210f-4f88-a777-82693acbdf6a"). InnerVolumeSpecName "kube-api-access-zr9qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:03:32.506404 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.506318 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/536038ed-210f-4f88-a777-82693acbdf6a-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:03:32.506404 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.506344 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zr9qj\" (UniqueName: \"kubernetes.io/projected/536038ed-210f-4f88-a777-82693acbdf6a-kube-api-access-zr9qj\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:03:32.506404 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.506353 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8059c2bc-282f-4bf5-8240-81218595b6e4-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:03:32.506404 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:32.506364 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh885\" (UniqueName: \"kubernetes.io/projected/8059c2bc-282f-4bf5-8240-81218595b6e4-kube-api-access-xh885\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:03:33.082480 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.082443 2564 generic.go:356] "Generic (PLEG): container finished" podID="8059c2bc-282f-4bf5-8240-81218595b6e4" containerID="d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f" exitCode=0 Jul 24 02:03:33.082890 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.082490 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" Jul 24 02:03:33.082890 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.082526 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" event={"ID":"8059c2bc-282f-4bf5-8240-81218595b6e4","Type":"ContainerDied","Data":"d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f"} Jul 24 02:03:33.082890 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.082575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f" event={"ID":"8059c2bc-282f-4bf5-8240-81218595b6e4","Type":"ContainerDied","Data":"1229725ee0239fbb27119cee56afc1a9eaebc6268d771456a48e34bfd3620d44"} Jul 24 02:03:33.082890 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.082604 2564 scope.go:117] "RemoveContainer" containerID="d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f" Jul 24 02:03:33.083842 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.083824 2564 generic.go:356] "Generic (PLEG): container finished" podID="536038ed-210f-4f88-a777-82693acbdf6a" containerID="054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691" exitCode=0 Jul 24 02:03:33.083896 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.083866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" event={"ID":"536038ed-210f-4f88-a777-82693acbdf6a","Type":"ContainerDied","Data":"054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691"} Jul 24 02:03:33.083933 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.083902 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" event={"ID":"536038ed-210f-4f88-a777-82693acbdf6a","Type":"ContainerDied","Data":"ea504aebc97f4b186391aa1bf539ffd95df1bdb1757ab9c50751ee57d6837154"} Jul 24 02:03:33.083933 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.083869 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m" Jul 24 02:03:33.091170 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.091154 2564 scope.go:117] "RemoveContainer" containerID="d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f" Jul 24 02:03:33.091426 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:03:33.091406 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f\": container with ID starting with d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f not found: ID does not exist" containerID="d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f" Jul 24 02:03:33.091501 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.091435 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f"} err="failed to get container status \"d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f\": rpc error: code = NotFound desc = could not find container \"d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f\": container with ID starting with d9416a0dada4ba0dcc76ca621304ea3c5bbee6c5e88de10ad15e5289fd813a2f not found: ID does not exist" Jul 24 02:03:33.091501 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.091450 2564 scope.go:117] "RemoveContainer" containerID="054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691" Jul 24 02:03:33.098837 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.098821 2564 scope.go:117] "RemoveContainer" containerID="054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691" Jul 24 02:03:33.099068 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:03:33.099047 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691\": container with ID starting with 054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691 not found: ID does not exist" containerID="054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691" Jul 24 02:03:33.099157 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.099079 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691"} err="failed to get container status \"054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691\": rpc error: code = NotFound desc = could not find container \"054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691\": container with ID starting with 054663f8b55e2ba9c2f16c0780739c91757731a3db337624a2ebf00e05ba2691 not found: ID does not exist" Jul 24 02:03:33.110985 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.108740 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f"] Jul 24 02:03:33.115202 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.115179 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d043f7ca-5776b88b94-bsl6f"] Jul 24 02:03:33.128274 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.128249 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m"] Jul 24 02:03:33.132730 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.132707 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d043f7ca-94d7fdcbd-v4p8m"] Jul 24 02:03:33.991870 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.991835 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="536038ed-210f-4f88-a777-82693acbdf6a" path="/var/lib/kubelet/pods/536038ed-210f-4f88-a777-82693acbdf6a/volumes" Jul 24 02:03:33.992216 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:03:33.992202 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8059c2bc-282f-4bf5-8240-81218595b6e4" path="/var/lib/kubelet/pods/8059c2bc-282f-4bf5-8240-81218595b6e4/volumes" Jul 24 02:04:51.961519 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:04:51.961481 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:04:51.969729 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:04:51.969703 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:05:27.764873 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.764840 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw"] Jul 24 02:05:27.765361 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.765117 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8059c2bc-282f-4bf5-8240-81218595b6e4" containerName="payload-processing" Jul 24 02:05:27.765361 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.765140 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8059c2bc-282f-4bf5-8240-81218595b6e4" containerName="payload-processing" Jul 24 02:05:27.765361 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.765158 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536038ed-210f-4f88-a777-82693acbdf6a" containerName="payload-pre-processing" Jul 24 02:05:27.765361 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.765164 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="536038ed-210f-4f88-a777-82693acbdf6a" containerName="payload-pre-processing" Jul 24 02:05:27.765361 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.765216 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8059c2bc-282f-4bf5-8240-81218595b6e4" containerName="payload-processing" Jul 24 02:05:27.765361 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.765224 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="536038ed-210f-4f88-a777-82693acbdf6a" containerName="payload-pre-processing" Jul 24 02:05:27.767957 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.767940 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:27.770716 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.770695 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b9895f-dockercfg-nckn9\"" Jul 24 02:05:27.770843 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.770775 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b9895f\"" Jul 24 02:05:27.779443 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.779422 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw"] Jul 24 02:05:27.800355 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.800335 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr"] Jul 24 02:05:27.803369 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.803354 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:27.814563 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.814542 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr"] Jul 24 02:05:27.867461 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.867427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a48ffb2c-16b7-458e-bf67-fef898f5f43d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:27.867608 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.867470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5scf2\" (UniqueName: \"kubernetes.io/projected/a48ffb2c-16b7-458e-bf67-fef898f5f43d-kube-api-access-5scf2\") pod \"payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:27.968787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.968749 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t29zz\" (UniqueName: \"kubernetes.io/projected/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-kube-api-access-t29zz\") pod \"payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:27.968944 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.968798 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:27.968944 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.968857 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a48ffb2c-16b7-458e-bf67-fef898f5f43d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:27.968944 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.968920 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5scf2\" (UniqueName: \"kubernetes.io/projected/a48ffb2c-16b7-458e-bf67-fef898f5f43d-kube-api-access-5scf2\") pod \"payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:27.969551 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.969533 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a48ffb2c-16b7-458e-bf67-fef898f5f43d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:27.977022 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:27.977000 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5scf2\" (UniqueName: \"kubernetes.io/projected/a48ffb2c-16b7-458e-bf67-fef898f5f43d-kube-api-access-5scf2\") pod \"payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:28.070322 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.070242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t29zz\" (UniqueName: \"kubernetes.io/projected/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-kube-api-access-t29zz\") pod \"payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:28.070322 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.070282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:28.070904 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.070886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:28.077680 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.077652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t29zz\" (UniqueName: \"kubernetes.io/projected/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-kube-api-access-t29zz\") pod \"payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:28.077777 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.077732 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:28.112043 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.111999 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:28.216462 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.216433 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw"] Jul 24 02:05:28.218736 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:05:28.218708 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48ffb2c_16b7_458e_bf67_fef898f5f43d.slice/crio-e67d04d2f5a5c7dca0ee0e7c3658917ff57e4d548fcda013115cccb9c5f92921 WatchSource:0}: Error finding container e67d04d2f5a5c7dca0ee0e7c3658917ff57e4d548fcda013115cccb9c5f92921: Status 404 returned error can't find the container with id e67d04d2f5a5c7dca0ee0e7c3658917ff57e4d548fcda013115cccb9c5f92921 Jul 24 02:05:28.249401 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.249383 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr"] Jul 24 02:05:28.255152 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:05:28.255110 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e9c0b7_0b2e_4a0f_ada5_14dd14766169.slice/crio-92a66fd034251897bb24bc4e23d0679e0d6d047c96e97b8e540814160c8bb2ef WatchSource:0}: Error finding container 92a66fd034251897bb24bc4e23d0679e0d6d047c96e97b8e540814160c8bb2ef: Status 404 returned error can't find the container with id 92a66fd034251897bb24bc4e23d0679e0d6d047c96e97b8e540814160c8bb2ef Jul 24 02:05:28.553796 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.553761 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" event={"ID":"53e9c0b7-0b2e-4a0f-ada5-14dd14766169","Type":"ContainerStarted","Data":"5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009"} Jul 24 02:05:28.553993 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.553822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" event={"ID":"53e9c0b7-0b2e-4a0f-ada5-14dd14766169","Type":"ContainerStarted","Data":"92a66fd034251897bb24bc4e23d0679e0d6d047c96e97b8e540814160c8bb2ef"} Jul 24 02:05:28.553993 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.553850 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:28.555241 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.555209 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" event={"ID":"a48ffb2c-16b7-458e-bf67-fef898f5f43d","Type":"ContainerStarted","Data":"3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17"} Jul 24 02:05:28.555360 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.555251 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" event={"ID":"a48ffb2c-16b7-458e-bf67-fef898f5f43d","Type":"ContainerStarted","Data":"e67d04d2f5a5c7dca0ee0e7c3658917ff57e4d548fcda013115cccb9c5f92921"} Jul 24 02:05:28.555360 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.555271 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:28.572034 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.571980 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" podStartSLOduration=1.5719613620000001 podStartE2EDuration="1.571961362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:05:28.571044978 +0000 UTC m=+2137.036179396" watchObservedRunningTime="2026-07-24 02:05:28.571961362 +0000 UTC m=+2137.037095781" Jul 24 02:05:28.587672 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:28.587626 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" podStartSLOduration=1.587608558 podStartE2EDuration="1.587608558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:05:28.586818702 +0000 UTC m=+2137.051953118" watchObservedRunningTime="2026-07-24 02:05:28.587608558 +0000 UTC m=+2137.052742977" Jul 24 02:05:35.712875 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.712836 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js"] Jul 24 02:05:35.716236 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.716219 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.718868 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.718849 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-2df67f-dockercfg-r5pk4\"" Jul 24 02:05:35.718997 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.718925 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-2df67f\"" Jul 24 02:05:35.722834 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.722813 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a9fef-c7f9-490d-bb4d-9069680574ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.722936 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.722858 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d728g\" (UniqueName: \"kubernetes.io/projected/830a9fef-c7f9-490d-bb4d-9069680574ef-kube-api-access-d728g\") pod \"payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.727189 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.727170 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js"] Jul 24 02:05:35.742391 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.742365 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w"] Jul 24 02:05:35.745328 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.745313 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:35.757861 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.757840 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w"] Jul 24 02:05:35.823709 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.823675 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-262m6\" (UniqueName: \"kubernetes.io/projected/b2222fbf-98bc-40a6-b1b2-3824e2511253-kube-api-access-262m6\") pod \"payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:35.823827 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.823791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a9fef-c7f9-490d-bb4d-9069680574ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.823875 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.823838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d728g\" (UniqueName: \"kubernetes.io/projected/830a9fef-c7f9-490d-bb4d-9069680574ef-kube-api-access-d728g\") pod \"payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.823911 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.823891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2222fbf-98bc-40a6-b1b2-3824e2511253-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:35.824420 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.824399 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a9fef-c7f9-490d-bb4d-9069680574ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.835081 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.835058 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d728g\" (UniqueName: \"kubernetes.io/projected/830a9fef-c7f9-490d-bb4d-9069680574ef-kube-api-access-d728g\") pod \"payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:35.925313 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.925281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2222fbf-98bc-40a6-b1b2-3824e2511253-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:35.925313 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.925314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-262m6\" (UniqueName: \"kubernetes.io/projected/b2222fbf-98bc-40a6-b1b2-3824e2511253-kube-api-access-262m6\") pod \"payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:35.926021 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.926000 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2222fbf-98bc-40a6-b1b2-3824e2511253-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:35.933211 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:35.933185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-262m6\" (UniqueName: \"kubernetes.io/projected/b2222fbf-98bc-40a6-b1b2-3824e2511253-kube-api-access-262m6\") pod \"payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:36.026468 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.026384 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:36.055386 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.055355 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:36.164195 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.164169 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js"] Jul 24 02:05:36.165907 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:05:36.165879 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830a9fef_c7f9_490d_bb4d_9069680574ef.slice/crio-0d690ac88feb5ba6222d8d218367a89939d48b6cbbea2225ef2dcd116088d959 WatchSource:0}: Error finding container 0d690ac88feb5ba6222d8d218367a89939d48b6cbbea2225ef2dcd116088d959: Status 404 returned error can't find the container with id 0d690ac88feb5ba6222d8d218367a89939d48b6cbbea2225ef2dcd116088d959 Jul 24 02:05:36.199911 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.199884 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w"] Jul 24 02:05:36.203058 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:05:36.203032 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2222fbf_98bc_40a6_b1b2_3824e2511253.slice/crio-a46d63c5dbd75be89d99f3f2fa55f23cd0f1407dadb824ed25ca43ad922bfc82 WatchSource:0}: Error finding container a46d63c5dbd75be89d99f3f2fa55f23cd0f1407dadb824ed25ca43ad922bfc82: Status 404 returned error can't find the container with id a46d63c5dbd75be89d99f3f2fa55f23cd0f1407dadb824ed25ca43ad922bfc82 Jul 24 02:05:36.594138 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.594091 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" event={"ID":"830a9fef-c7f9-490d-bb4d-9069680574ef","Type":"ContainerStarted","Data":"546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e"} Jul 24 02:05:36.594323 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.594185 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" event={"ID":"830a9fef-c7f9-490d-bb4d-9069680574ef","Type":"ContainerStarted","Data":"0d690ac88feb5ba6222d8d218367a89939d48b6cbbea2225ef2dcd116088d959"} Jul 24 02:05:36.594323 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.594222 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:36.595551 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.595529 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" event={"ID":"b2222fbf-98bc-40a6-b1b2-3824e2511253","Type":"ContainerStarted","Data":"eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845"} Jul 24 02:05:36.595663 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.595566 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" event={"ID":"b2222fbf-98bc-40a6-b1b2-3824e2511253","Type":"ContainerStarted","Data":"a46d63c5dbd75be89d99f3f2fa55f23cd0f1407dadb824ed25ca43ad922bfc82"} Jul 24 02:05:36.595663 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.595601 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:05:36.610515 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.610467 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" podStartSLOduration=1.610452998 podStartE2EDuration="1.610452998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:05:36.609540029 +0000 UTC m=+2145.074674468" watchObservedRunningTime="2026-07-24 02:05:36.610452998 +0000 UTC m=+2145.075587415" Jul 24 02:05:36.626202 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:36.626159 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" podStartSLOduration=1.626140498 podStartE2EDuration="1.626140498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:05:36.624784606 +0000 UTC m=+2145.089919045" watchObservedRunningTime="2026-07-24 02:05:36.626140498 +0000 UTC m=+2145.091274914" Jul 24 02:05:39.561346 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:39.561312 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:05:39.561733 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:39.561382 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:05:47.602089 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:47.602015 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:05:47.602089 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:05:47.602085 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:07:00.505996 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.505956 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr"] Jul 24 02:07:00.506551 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.506256 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" podUID="53e9c0b7-0b2e-4a0f-ada5-14dd14766169" containerName="payload-processing" containerID="cri-o://5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009" gracePeriod=30 Jul 24 02:07:00.509307 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.509279 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw"] Jul 24 02:07:00.509643 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.509592 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" podUID="a48ffb2c-16b7-458e-bf67-fef898f5f43d" containerName="payload-pre-processing" containerID="cri-o://3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17" gracePeriod=30 Jul 24 02:07:00.787870 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.787845 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:07:00.790756 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.790737 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:07:00.953180 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.953142 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t29zz\" (UniqueName: \"kubernetes.io/projected/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-kube-api-access-t29zz\") pod \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " Jul 24 02:07:00.953377 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.953226 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5scf2\" (UniqueName: \"kubernetes.io/projected/a48ffb2c-16b7-458e-bf67-fef898f5f43d-kube-api-access-5scf2\") pod \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " Jul 24 02:07:00.953377 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.953257 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-plugins-config-volume\") pod \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\" (UID: \"53e9c0b7-0b2e-4a0f-ada5-14dd14766169\") " Jul 24 02:07:00.953377 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.953311 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a48ffb2c-16b7-458e-bf67-fef898f5f43d-plugins-config-volume\") pod \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\" (UID: \"a48ffb2c-16b7-458e-bf67-fef898f5f43d\") " Jul 24 02:07:00.953687 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.953649 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "53e9c0b7-0b2e-4a0f-ada5-14dd14766169" (UID: "53e9c0b7-0b2e-4a0f-ada5-14dd14766169"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:07:00.953785 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.953688 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48ffb2c-16b7-458e-bf67-fef898f5f43d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a48ffb2c-16b7-458e-bf67-fef898f5f43d" (UID: "a48ffb2c-16b7-458e-bf67-fef898f5f43d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:07:00.955507 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.955477 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a48ffb2c-16b7-458e-bf67-fef898f5f43d-kube-api-access-5scf2" (OuterVolumeSpecName: "kube-api-access-5scf2") pod "a48ffb2c-16b7-458e-bf67-fef898f5f43d" (UID: "a48ffb2c-16b7-458e-bf67-fef898f5f43d"). InnerVolumeSpecName "kube-api-access-5scf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:07:00.955667 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.955647 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-kube-api-access-t29zz" (OuterVolumeSpecName: "kube-api-access-t29zz") pod "53e9c0b7-0b2e-4a0f-ada5-14dd14766169" (UID: "53e9c0b7-0b2e-4a0f-ada5-14dd14766169"). InnerVolumeSpecName "kube-api-access-t29zz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:07:00.957264 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.957237 2564 generic.go:356] "Generic (PLEG): container finished" podID="53e9c0b7-0b2e-4a0f-ada5-14dd14766169" containerID="5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009" exitCode=0 Jul 24 02:07:00.957367 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.957289 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" Jul 24 02:07:00.957367 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.957316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" event={"ID":"53e9c0b7-0b2e-4a0f-ada5-14dd14766169","Type":"ContainerDied","Data":"5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009"} Jul 24 02:07:00.957482 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.957375 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr" event={"ID":"53e9c0b7-0b2e-4a0f-ada5-14dd14766169","Type":"ContainerDied","Data":"92a66fd034251897bb24bc4e23d0679e0d6d047c96e97b8e540814160c8bb2ef"} Jul 24 02:07:00.957482 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.957399 2564 scope.go:117] "RemoveContainer" containerID="5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009" Jul 24 02:07:00.958560 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.958540 2564 generic.go:356] "Generic (PLEG): container finished" podID="a48ffb2c-16b7-458e-bf67-fef898f5f43d" containerID="3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17" exitCode=0 Jul 24 02:07:00.958651 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.958599 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" Jul 24 02:07:00.958651 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.958623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" event={"ID":"a48ffb2c-16b7-458e-bf67-fef898f5f43d","Type":"ContainerDied","Data":"3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17"} Jul 24 02:07:00.958752 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.958672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw" event={"ID":"a48ffb2c-16b7-458e-bf67-fef898f5f43d","Type":"ContainerDied","Data":"e67d04d2f5a5c7dca0ee0e7c3658917ff57e4d548fcda013115cccb9c5f92921"} Jul 24 02:07:00.965445 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.965426 2564 scope.go:117] "RemoveContainer" containerID="5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009" Jul 24 02:07:00.965704 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:07:00.965684 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009\": container with ID starting with 5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009 not found: ID does not exist" containerID="5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009" Jul 24 02:07:00.965757 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.965714 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009"} err="failed to get container status \"5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009\": rpc error: code = NotFound desc = could not find container \"5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009\": container with ID starting with 5aa01afb1c5004b5eb02fcf88b11397c09245f65fdfe2515bea887e14209e009 not found: ID does not exist" Jul 24 02:07:00.965757 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.965734 2564 scope.go:117] "RemoveContainer" containerID="3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17" Jul 24 02:07:00.973883 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.973839 2564 scope.go:117] "RemoveContainer" containerID="3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17" Jul 24 02:07:00.974114 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:07:00.974094 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17\": container with ID starting with 3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17 not found: ID does not exist" containerID="3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17" Jul 24 02:07:00.974232 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.974119 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17"} err="failed to get container status \"3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17\": rpc error: code = NotFound desc = could not find container \"3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17\": container with ID starting with 3cff8335bcce9eb3fc5ebc64b7d723101935fbe3a93659104c041f8aa3cc6c17 not found: ID does not exist" Jul 24 02:07:00.985474 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.985452 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr"] Jul 24 02:07:00.991269 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:00.991240 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b9895f-58f74dcb4-rqmmr"] Jul 24 02:07:01.003232 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.003209 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw"] Jul 24 02:07:01.010622 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.010599 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b9895f-64666f895b-7htlw"] Jul 24 02:07:01.054112 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.054035 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5scf2\" (UniqueName: \"kubernetes.io/projected/a48ffb2c-16b7-458e-bf67-fef898f5f43d-kube-api-access-5scf2\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:01.054112 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.054063 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:01.054112 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.054074 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a48ffb2c-16b7-458e-bf67-fef898f5f43d-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:01.054112 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.054084 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t29zz\" (UniqueName: \"kubernetes.io/projected/53e9c0b7-0b2e-4a0f-ada5-14dd14766169-kube-api-access-t29zz\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:01.992196 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.992164 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53e9c0b7-0b2e-4a0f-ada5-14dd14766169" path="/var/lib/kubelet/pods/53e9c0b7-0b2e-4a0f-ada5-14dd14766169/volumes" Jul 24 02:07:01.992572 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:01.992527 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48ffb2c-16b7-458e-bf67-fef898f5f43d" path="/var/lib/kubelet/pods/a48ffb2c-16b7-458e-bf67-fef898f5f43d/volumes" Jul 24 02:07:22.686376 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:22.686292 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w"] Jul 24 02:07:22.686824 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:22.686577 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" podUID="b2222fbf-98bc-40a6-b1b2-3824e2511253" containerName="payload-processing" containerID="cri-o://eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845" gracePeriod=30 Jul 24 02:07:22.696719 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:22.696572 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js"] Jul 24 02:07:22.696836 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:22.696765 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" podUID="830a9fef-c7f9-490d-bb4d-9069680574ef" containerName="payload-pre-processing" containerID="cri-o://546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e" gracePeriod=30 Jul 24 02:07:22.946506 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:22.946444 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:07:22.969365 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:22.969342 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:07:23.028366 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.028335 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a9fef-c7f9-490d-bb4d-9069680574ef-plugins-config-volume\") pod \"830a9fef-c7f9-490d-bb4d-9069680574ef\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " Jul 24 02:07:23.028537 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.028376 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-262m6\" (UniqueName: \"kubernetes.io/projected/b2222fbf-98bc-40a6-b1b2-3824e2511253-kube-api-access-262m6\") pod \"b2222fbf-98bc-40a6-b1b2-3824e2511253\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " Jul 24 02:07:23.028537 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.028410 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d728g\" (UniqueName: \"kubernetes.io/projected/830a9fef-c7f9-490d-bb4d-9069680574ef-kube-api-access-d728g\") pod \"830a9fef-c7f9-490d-bb4d-9069680574ef\" (UID: \"830a9fef-c7f9-490d-bb4d-9069680574ef\") " Jul 24 02:07:23.028537 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.028498 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2222fbf-98bc-40a6-b1b2-3824e2511253-plugins-config-volume\") pod \"b2222fbf-98bc-40a6-b1b2-3824e2511253\" (UID: \"b2222fbf-98bc-40a6-b1b2-3824e2511253\") " Jul 24 02:07:23.028801 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.028777 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/830a9fef-c7f9-490d-bb4d-9069680574ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "830a9fef-c7f9-490d-bb4d-9069680574ef" (UID: "830a9fef-c7f9-490d-bb4d-9069680574ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:07:23.028860 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.028821 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2222fbf-98bc-40a6-b1b2-3824e2511253-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2222fbf-98bc-40a6-b1b2-3824e2511253" (UID: "b2222fbf-98bc-40a6-b1b2-3824e2511253"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:07:23.030466 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.030445 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/830a9fef-c7f9-490d-bb4d-9069680574ef-kube-api-access-d728g" (OuterVolumeSpecName: "kube-api-access-d728g") pod "830a9fef-c7f9-490d-bb4d-9069680574ef" (UID: "830a9fef-c7f9-490d-bb4d-9069680574ef"). InnerVolumeSpecName "kube-api-access-d728g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:07:23.030536 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.030516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2222fbf-98bc-40a6-b1b2-3824e2511253-kube-api-access-262m6" (OuterVolumeSpecName: "kube-api-access-262m6") pod "b2222fbf-98bc-40a6-b1b2-3824e2511253" (UID: "b2222fbf-98bc-40a6-b1b2-3824e2511253"). InnerVolumeSpecName "kube-api-access-262m6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:07:23.051322 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.051300 2564 generic.go:356] "Generic (PLEG): container finished" podID="b2222fbf-98bc-40a6-b1b2-3824e2511253" containerID="eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845" exitCode=0 Jul 24 02:07:23.051415 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.051342 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" Jul 24 02:07:23.051415 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.051390 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" event={"ID":"b2222fbf-98bc-40a6-b1b2-3824e2511253","Type":"ContainerDied","Data":"eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845"} Jul 24 02:07:23.051511 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.051448 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w" event={"ID":"b2222fbf-98bc-40a6-b1b2-3824e2511253","Type":"ContainerDied","Data":"a46d63c5dbd75be89d99f3f2fa55f23cd0f1407dadb824ed25ca43ad922bfc82"} Jul 24 02:07:23.051511 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.051481 2564 scope.go:117] "RemoveContainer" containerID="eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845" Jul 24 02:07:23.052636 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.052607 2564 generic.go:356] "Generic (PLEG): container finished" podID="830a9fef-c7f9-490d-bb4d-9069680574ef" containerID="546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e" exitCode=0 Jul 24 02:07:23.052735 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.052639 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" event={"ID":"830a9fef-c7f9-490d-bb4d-9069680574ef","Type":"ContainerDied","Data":"546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e"} Jul 24 02:07:23.052735 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.052646 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" Jul 24 02:07:23.052735 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.052678 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js" event={"ID":"830a9fef-c7f9-490d-bb4d-9069680574ef","Type":"ContainerDied","Data":"0d690ac88feb5ba6222d8d218367a89939d48b6cbbea2225ef2dcd116088d959"} Jul 24 02:07:23.060667 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.060642 2564 scope.go:117] "RemoveContainer" containerID="eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845" Jul 24 02:07:23.061141 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:07:23.061098 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845\": container with ID starting with eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845 not found: ID does not exist" containerID="eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845" Jul 24 02:07:23.061236 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.061150 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845"} err="failed to get container status \"eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845\": rpc error: code = NotFound desc = could not find container \"eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845\": container with ID starting with eea3fb2a1e075e3f62d268fcc2fbc40b4fded0fc99d751ad8a558b1f29209845 not found: ID does not exist" Jul 24 02:07:23.061236 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.061173 2564 scope.go:117] "RemoveContainer" containerID="546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e" Jul 24 02:07:23.070243 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.070225 2564 scope.go:117] "RemoveContainer" containerID="546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e" Jul 24 02:07:23.070502 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:07:23.070482 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e\": container with ID starting with 546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e not found: ID does not exist" containerID="546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e" Jul 24 02:07:23.070567 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.070513 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e"} err="failed to get container status \"546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e\": rpc error: code = NotFound desc = could not find container \"546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e\": container with ID starting with 546f3df774a59efd9ee965f90b8bc40229b2deb58d8f35393438daefdc7ee26e not found: ID does not exist" Jul 24 02:07:23.077507 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.077486 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js"] Jul 24 02:07:23.080996 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.080975 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-2df67f-568774559b-679js"] Jul 24 02:07:23.091296 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.091275 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w"] Jul 24 02:07:23.095221 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.095201 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2df67f-b67687758-2bj4w"] Jul 24 02:07:23.129283 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.129250 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/830a9fef-c7f9-490d-bb4d-9069680574ef-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:23.129283 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.129280 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-262m6\" (UniqueName: \"kubernetes.io/projected/b2222fbf-98bc-40a6-b1b2-3824e2511253-kube-api-access-262m6\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:23.129436 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.129296 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d728g\" (UniqueName: \"kubernetes.io/projected/830a9fef-c7f9-490d-bb4d-9069680574ef-kube-api-access-d728g\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:23.129436 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.129310 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2222fbf-98bc-40a6-b1b2-3824e2511253-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:07:23.991854 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.991777 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830a9fef-c7f9-490d-bb4d-9069680574ef" path="/var/lib/kubelet/pods/830a9fef-c7f9-490d-bb4d-9069680574ef/volumes" Jul 24 02:07:23.992216 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:23.992113 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2222fbf-98bc-40a6-b1b2-3824e2511253" path="/var/lib/kubelet/pods/b2222fbf-98bc-40a6-b1b2-3824e2511253/volumes" Jul 24 02:07:46.519085 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519049 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl"] Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519353 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48ffb2c-16b7-458e-bf67-fef898f5f43d" containerName="payload-pre-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519365 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48ffb2c-16b7-458e-bf67-fef898f5f43d" containerName="payload-pre-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519375 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830a9fef-c7f9-490d-bb4d-9069680574ef" containerName="payload-pre-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519380 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="830a9fef-c7f9-490d-bb4d-9069680574ef" containerName="payload-pre-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519397 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2222fbf-98bc-40a6-b1b2-3824e2511253" containerName="payload-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519403 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2222fbf-98bc-40a6-b1b2-3824e2511253" containerName="payload-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519415 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e9c0b7-0b2e-4a0f-ada5-14dd14766169" containerName="payload-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519421 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e9c0b7-0b2e-4a0f-ada5-14dd14766169" containerName="payload-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519470 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="830a9fef-c7f9-490d-bb4d-9069680574ef" containerName="payload-pre-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519478 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48ffb2c-16b7-458e-bf67-fef898f5f43d" containerName="payload-pre-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519484 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e9c0b7-0b2e-4a0f-ada5-14dd14766169" containerName="payload-processing" Jul 24 02:07:46.519579 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.519489 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2222fbf-98bc-40a6-b1b2-3824e2511253" containerName="payload-processing" Jul 24 02:07:46.522449 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.522431 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.525080 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.525056 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a955c2-dockercfg-2ccwv\"" Jul 24 02:07:46.525344 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.525328 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a955c2\"" Jul 24 02:07:46.532322 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.532300 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl"] Jul 24 02:07:46.544622 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.544599 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2"] Jul 24 02:07:46.548203 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.548185 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.557866 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.557846 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2"] Jul 24 02:07:46.625367 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.625332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq5mr\" (UniqueName: \"kubernetes.io/projected/17ccca6c-686a-4910-9d63-4128470f473c-kube-api-access-kq5mr\") pod \"payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.625541 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.625372 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9g8r\" (UniqueName: \"kubernetes.io/projected/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-kube-api-access-k9g8r\") pod \"payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.625541 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.625453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.625541 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.625492 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ccca6c-686a-4910-9d63-4128470f473c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.726287 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.726252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ccca6c-686a-4910-9d63-4128470f473c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.726456 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.726314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq5mr\" (UniqueName: \"kubernetes.io/projected/17ccca6c-686a-4910-9d63-4128470f473c-kube-api-access-kq5mr\") pod \"payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.726456 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.726352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9g8r\" (UniqueName: \"kubernetes.io/projected/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-kube-api-access-k9g8r\") pod \"payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.726531 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.726492 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.727005 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.726988 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ccca6c-686a-4910-9d63-4128470f473c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.727175 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.727158 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.738037 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.738009 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq5mr\" (UniqueName: \"kubernetes.io/projected/17ccca6c-686a-4910-9d63-4128470f473c-kube-api-access-kq5mr\") pod \"payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.738807 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.738779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9g8r\" (UniqueName: \"kubernetes.io/projected/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-kube-api-access-k9g8r\") pod \"payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.832782 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.832707 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:46.857944 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.857921 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:46.976600 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:46.976569 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl"] Jul 24 02:07:46.979797 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:07:46.979764 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b3feb5_c2ce_40f6_8345_afe2e13b9b46.slice/crio-a64ba8df1febd09b127d782aa1e1270f11574add819e95245a98df5f358391bc WatchSource:0}: Error finding container a64ba8df1febd09b127d782aa1e1270f11574add819e95245a98df5f358391bc: Status 404 returned error can't find the container with id a64ba8df1febd09b127d782aa1e1270f11574add819e95245a98df5f358391bc Jul 24 02:07:47.001372 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.001349 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2"] Jul 24 02:07:47.016184 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:07:47.016158 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ccca6c_686a_4910_9d63_4128470f473c.slice/crio-2af8e2dc84254d44e2bf3e1354d2d9c5f657f0b28c9e49f9b20648f68e37a533 WatchSource:0}: Error finding container 2af8e2dc84254d44e2bf3e1354d2d9c5f657f0b28c9e49f9b20648f68e37a533: Status 404 returned error can't find the container with id 2af8e2dc84254d44e2bf3e1354d2d9c5f657f0b28c9e49f9b20648f68e37a533 Jul 24 02:07:47.157691 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.157646 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" event={"ID":"47b3feb5-c2ce-40f6-8345-afe2e13b9b46","Type":"ContainerStarted","Data":"996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177"} Jul 24 02:07:47.157886 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.157719 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" event={"ID":"47b3feb5-c2ce-40f6-8345-afe2e13b9b46","Type":"ContainerStarted","Data":"a64ba8df1febd09b127d782aa1e1270f11574add819e95245a98df5f358391bc"} Jul 24 02:07:47.157886 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.157870 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:07:47.159297 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.159268 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" event={"ID":"17ccca6c-686a-4910-9d63-4128470f473c","Type":"ContainerStarted","Data":"a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a"} Jul 24 02:07:47.159428 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.159312 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" event={"ID":"17ccca6c-686a-4910-9d63-4128470f473c","Type":"ContainerStarted","Data":"2af8e2dc84254d44e2bf3e1354d2d9c5f657f0b28c9e49f9b20648f68e37a533"} Jul 24 02:07:47.159428 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.159386 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:47.174632 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.174585 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" podStartSLOduration=1.174565133 podStartE2EDuration="1.174565133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:07:47.174017741 +0000 UTC m=+2275.639152159" watchObservedRunningTime="2026-07-24 02:07:47.174565133 +0000 UTC m=+2275.639699553" Jul 24 02:07:47.193816 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:47.193778 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" podStartSLOduration=1.193763935 podStartE2EDuration="1.193763935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:07:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:07:47.193430451 +0000 UTC m=+2275.658564906" watchObservedRunningTime="2026-07-24 02:07:47.193763935 +0000 UTC m=+2275.658898352" Jul 24 02:07:58.165521 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:58.165487 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:07:58.166015 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:07:58.165558 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:08:41.365919 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.365879 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2"] Jul 24 02:08:41.366546 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.366085 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" podUID="17ccca6c-686a-4910-9d63-4128470f473c" containerName="payload-processing" containerID="cri-o://a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a" gracePeriod=30 Jul 24 02:08:41.374427 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.374399 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl"] Jul 24 02:08:41.374662 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.374636 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" podUID="47b3feb5-c2ce-40f6-8345-afe2e13b9b46" containerName="payload-pre-processing" containerID="cri-o://996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177" gracePeriod=30 Jul 24 02:08:41.628993 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.628968 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:08:41.633591 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.633572 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:08:41.682732 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.682706 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-plugins-config-volume\") pod \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " Jul 24 02:08:41.682897 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.682820 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq5mr\" (UniqueName: \"kubernetes.io/projected/17ccca6c-686a-4910-9d63-4128470f473c-kube-api-access-kq5mr\") pod \"17ccca6c-686a-4910-9d63-4128470f473c\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " Jul 24 02:08:41.682897 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.682838 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9g8r\" (UniqueName: \"kubernetes.io/projected/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-kube-api-access-k9g8r\") pod \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\" (UID: \"47b3feb5-c2ce-40f6-8345-afe2e13b9b46\") " Jul 24 02:08:41.682897 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.682860 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ccca6c-686a-4910-9d63-4128470f473c-plugins-config-volume\") pod \"17ccca6c-686a-4910-9d63-4128470f473c\" (UID: \"17ccca6c-686a-4910-9d63-4128470f473c\") " Jul 24 02:08:41.683187 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.683114 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47b3feb5-c2ce-40f6-8345-afe2e13b9b46" (UID: "47b3feb5-c2ce-40f6-8345-afe2e13b9b46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:08:41.683325 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.683303 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17ccca6c-686a-4910-9d63-4128470f473c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "17ccca6c-686a-4910-9d63-4128470f473c" (UID: "17ccca6c-686a-4910-9d63-4128470f473c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:08:41.684844 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.684821 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17ccca6c-686a-4910-9d63-4128470f473c-kube-api-access-kq5mr" (OuterVolumeSpecName: "kube-api-access-kq5mr") pod "17ccca6c-686a-4910-9d63-4128470f473c" (UID: "17ccca6c-686a-4910-9d63-4128470f473c"). InnerVolumeSpecName "kube-api-access-kq5mr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:08:41.684894 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.684832 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-kube-api-access-k9g8r" (OuterVolumeSpecName: "kube-api-access-k9g8r") pod "47b3feb5-c2ce-40f6-8345-afe2e13b9b46" (UID: "47b3feb5-c2ce-40f6-8345-afe2e13b9b46"). InnerVolumeSpecName "kube-api-access-k9g8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:08:41.783968 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.783933 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq5mr\" (UniqueName: \"kubernetes.io/projected/17ccca6c-686a-4910-9d63-4128470f473c-kube-api-access-kq5mr\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:08:41.783968 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.783963 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9g8r\" (UniqueName: \"kubernetes.io/projected/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-kube-api-access-k9g8r\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:08:41.783968 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.783976 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17ccca6c-686a-4910-9d63-4128470f473c-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:08:41.784220 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:41.783985 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47b3feb5-c2ce-40f6-8345-afe2e13b9b46-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:08:42.399234 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.399202 2564 generic.go:356] "Generic (PLEG): container finished" podID="17ccca6c-686a-4910-9d63-4128470f473c" containerID="a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a" exitCode=0 Jul 24 02:08:42.399706 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.399244 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" Jul 24 02:08:42.399706 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.399287 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" event={"ID":"17ccca6c-686a-4910-9d63-4128470f473c","Type":"ContainerDied","Data":"a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a"} Jul 24 02:08:42.399706 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.399348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2" event={"ID":"17ccca6c-686a-4910-9d63-4128470f473c","Type":"ContainerDied","Data":"2af8e2dc84254d44e2bf3e1354d2d9c5f657f0b28c9e49f9b20648f68e37a533"} Jul 24 02:08:42.399706 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.399383 2564 scope.go:117] "RemoveContainer" containerID="a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a" Jul 24 02:08:42.400731 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.400568 2564 generic.go:356] "Generic (PLEG): container finished" podID="47b3feb5-c2ce-40f6-8345-afe2e13b9b46" containerID="996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177" exitCode=0 Jul 24 02:08:42.400731 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.400630 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" event={"ID":"47b3feb5-c2ce-40f6-8345-afe2e13b9b46","Type":"ContainerDied","Data":"996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177"} Jul 24 02:08:42.400731 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.400639 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" Jul 24 02:08:42.400731 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.400674 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl" event={"ID":"47b3feb5-c2ce-40f6-8345-afe2e13b9b46","Type":"ContainerDied","Data":"a64ba8df1febd09b127d782aa1e1270f11574add819e95245a98df5f358391bc"} Jul 24 02:08:42.407871 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.407853 2564 scope.go:117] "RemoveContainer" containerID="a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a" Jul 24 02:08:42.408184 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:08:42.408159 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a\": container with ID starting with a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a not found: ID does not exist" containerID="a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a" Jul 24 02:08:42.408298 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.408192 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a"} err="failed to get container status \"a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a\": rpc error: code = NotFound desc = could not find container \"a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a\": container with ID starting with a99a3efc19c2e11d42bd9a6a582236c074cfd1c6193501d7484c14d74f2a794a not found: ID does not exist" Jul 24 02:08:42.408298 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.408209 2564 scope.go:117] "RemoveContainer" containerID="996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177" Jul 24 02:08:42.415592 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.415570 2564 scope.go:117] "RemoveContainer" containerID="996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177" Jul 24 02:08:42.415879 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:08:42.415858 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177\": container with ID starting with 996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177 not found: ID does not exist" containerID="996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177" Jul 24 02:08:42.415944 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.415887 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177"} err="failed to get container status \"996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177\": rpc error: code = NotFound desc = could not find container \"996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177\": container with ID starting with 996fbdc53f053614b6ec27b4bc6db1745880717b49ba7e1468446ff76da1a177 not found: ID does not exist" Jul 24 02:08:42.421553 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.421527 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2"] Jul 24 02:08:42.428504 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.428481 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a955c2-58589fbc48-t4sr2"] Jul 24 02:08:42.439344 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.439320 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl"] Jul 24 02:08:42.443259 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:42.443239 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a955c2-95b5c69b-v2hrl"] Jul 24 02:08:43.992305 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:43.992222 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17ccca6c-686a-4910-9d63-4128470f473c" path="/var/lib/kubelet/pods/17ccca6c-686a-4910-9d63-4128470f473c/volumes" Jul 24 02:08:43.992655 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:43.992635 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47b3feb5-c2ce-40f6-8345-afe2e13b9b46" path="/var/lib/kubelet/pods/47b3feb5-c2ce-40f6-8345-afe2e13b9b46/volumes" Jul 24 02:08:57.377905 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.377867 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz"] Jul 24 02:08:57.378310 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.378185 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47b3feb5-c2ce-40f6-8345-afe2e13b9b46" containerName="payload-pre-processing" Jul 24 02:08:57.378310 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.378198 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="47b3feb5-c2ce-40f6-8345-afe2e13b9b46" containerName="payload-pre-processing" Jul 24 02:08:57.378310 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.378215 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17ccca6c-686a-4910-9d63-4128470f473c" containerName="payload-processing" Jul 24 02:08:57.378310 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.378220 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17ccca6c-686a-4910-9d63-4128470f473c" containerName="payload-processing" Jul 24 02:08:57.378310 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.378273 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17ccca6c-686a-4910-9d63-4128470f473c" containerName="payload-processing" Jul 24 02:08:57.378310 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.378283 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="47b3feb5-c2ce-40f6-8345-afe2e13b9b46" containerName="payload-pre-processing" Jul 24 02:08:57.382615 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.382587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.385645 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.385621 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-6bcabd\"" Jul 24 02:08:57.399855 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.399825 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz"] Jul 24 02:08:57.401604 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.401571 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr"] Jul 24 02:08:57.405051 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.405029 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.407934 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.407915 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-6bcabd-dockercfg-ghdc6\"" Jul 24 02:08:57.415359 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.415333 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr"] Jul 24 02:08:57.529146 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.529088 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.529328 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.529180 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2789j\" (UniqueName: \"kubernetes.io/projected/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-kube-api-access-2789j\") pod \"payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.529328 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.529214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.529328 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.529274 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mlmx\" (UniqueName: \"kubernetes.io/projected/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-kube-api-access-5mlmx\") pod \"payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.629763 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.629666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2789j\" (UniqueName: \"kubernetes.io/projected/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-kube-api-access-2789j\") pod \"payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.629763 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.629720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.630002 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.629769 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mlmx\" (UniqueName: \"kubernetes.io/projected/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-kube-api-access-5mlmx\") pod \"payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.630002 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.629818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.630587 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.630562 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.630587 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.630580 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.642455 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.642433 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mlmx\" (UniqueName: \"kubernetes.io/projected/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-kube-api-access-5mlmx\") pod \"payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.642455 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.642443 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2789j\" (UniqueName: \"kubernetes.io/projected/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-kube-api-access-2789j\") pod \"payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.692374 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.692341 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:57.718631 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.718592 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:57.856633 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.856403 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz"] Jul 24 02:08:57.858908 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:08:57.858880 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod865b48f8_ce23_4b68_a33c_d8cbb0fdffda.slice/crio-1ccf4ffa5965c6d61805d8ab039a1dba00d165595437bb0f2d780f7b4fdc5aa8 WatchSource:0}: Error finding container 1ccf4ffa5965c6d61805d8ab039a1dba00d165595437bb0f2d780f7b4fdc5aa8: Status 404 returned error can't find the container with id 1ccf4ffa5965c6d61805d8ab039a1dba00d165595437bb0f2d780f7b4fdc5aa8 Jul 24 02:08:57.880571 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:57.880542 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr"] Jul 24 02:08:57.885034 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:08:57.885001 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f978d28_fd7e_45a6_98dd_d3c575dcd58e.slice/crio-fcdb7c456f52cab0dfe5f90b61115710bc71126857f6fb6262397c0d0615c07c WatchSource:0}: Error finding container fcdb7c456f52cab0dfe5f90b61115710bc71126857f6fb6262397c0d0615c07c: Status 404 returned error can't find the container with id fcdb7c456f52cab0dfe5f90b61115710bc71126857f6fb6262397c0d0615c07c Jul 24 02:08:58.468920 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.468880 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" event={"ID":"7f978d28-fd7e-45a6-98dd-d3c575dcd58e","Type":"ContainerStarted","Data":"82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c"} Jul 24 02:08:58.469406 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.468945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" event={"ID":"7f978d28-fd7e-45a6-98dd-d3c575dcd58e","Type":"ContainerStarted","Data":"fcdb7c456f52cab0dfe5f90b61115710bc71126857f6fb6262397c0d0615c07c"} Jul 24 02:08:58.469406 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.468990 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:08:58.470662 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.470638 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" event={"ID":"865b48f8-ce23-4b68-a33c-d8cbb0fdffda","Type":"ContainerStarted","Data":"bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243"} Jul 24 02:08:58.470796 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.470675 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" event={"ID":"865b48f8-ce23-4b68-a33c-d8cbb0fdffda","Type":"ContainerStarted","Data":"1ccf4ffa5965c6d61805d8ab039a1dba00d165595437bb0f2d780f7b4fdc5aa8"} Jul 24 02:08:58.470796 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.470758 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:08:58.488804 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.488757 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" podStartSLOduration=1.4887410939999999 podStartE2EDuration="1.488741094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:08:58.486744641 +0000 UTC m=+2346.951879081" watchObservedRunningTime="2026-07-24 02:08:58.488741094 +0000 UTC m=+2346.953875511" Jul 24 02:08:58.503306 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:08:58.503254 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" podStartSLOduration=1.5032363279999998 podStartE2EDuration="1.503236328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:08:58.502391496 +0000 UTC m=+2346.967525912" watchObservedRunningTime="2026-07-24 02:08:58.503236328 +0000 UTC m=+2346.968370774" Jul 24 02:09:05.985613 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:05.985573 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw"] Jul 24 02:09:05.989206 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:05.989176 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:05.991913 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:05.991890 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-a7fbc1\"" Jul 24 02:09:05.999879 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:05.999855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmqqq\" (UniqueName: \"kubernetes.io/projected/1188575b-2c3d-4bbb-877a-11f4ddd33629-kube-api-access-xmqqq\") pod \"payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:05.999990 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:05.999894 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1188575b-2c3d-4bbb-877a-11f4ddd33629-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:06.002789 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.002765 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw"] Jul 24 02:09:06.101162 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.101096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmqqq\" (UniqueName: \"kubernetes.io/projected/1188575b-2c3d-4bbb-877a-11f4ddd33629-kube-api-access-xmqqq\") pod \"payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:06.101162 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.101171 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1188575b-2c3d-4bbb-877a-11f4ddd33629-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:06.101910 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.101886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1188575b-2c3d-4bbb-877a-11f4ddd33629-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:06.119817 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.119779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmqqq\" (UniqueName: \"kubernetes.io/projected/1188575b-2c3d-4bbb-877a-11f4ddd33629-kube-api-access-xmqqq\") pod \"payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:06.129247 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.129223 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62"] Jul 24 02:09:06.132504 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.132489 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.135623 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.135606 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-a7fbc1-dockercfg-fhx97\"" Jul 24 02:09:06.149117 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.149091 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62"] Jul 24 02:09:06.201630 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.201590 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg6fg\" (UniqueName: \"kubernetes.io/projected/c014f80f-1324-4e56-b784-dd80d163e4c9-kube-api-access-gg6fg\") pod \"payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.201799 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.201653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c014f80f-1324-4e56-b784-dd80d163e4c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.300751 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.300654 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:06.301990 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.301968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg6fg\" (UniqueName: \"kubernetes.io/projected/c014f80f-1324-4e56-b784-dd80d163e4c9-kube-api-access-gg6fg\") pod \"payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.302068 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.302029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c014f80f-1324-4e56-b784-dd80d163e4c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.302660 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.302639 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c014f80f-1324-4e56-b784-dd80d163e4c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.312762 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.312736 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg6fg\" (UniqueName: \"kubernetes.io/projected/c014f80f-1324-4e56-b784-dd80d163e4c9-kube-api-access-gg6fg\") pod \"payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.442638 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.442596 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:06.463267 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.463244 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw"] Jul 24 02:09:06.466334 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:09:06.466304 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1188575b_2c3d_4bbb_877a_11f4ddd33629.slice/crio-ffa89f6c16250b61b0cf7d948eb4623ad89aaa68b20b5ed9945925364052d383 WatchSource:0}: Error finding container ffa89f6c16250b61b0cf7d948eb4623ad89aaa68b20b5ed9945925364052d383: Status 404 returned error can't find the container with id ffa89f6c16250b61b0cf7d948eb4623ad89aaa68b20b5ed9945925364052d383 Jul 24 02:09:06.508041 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.507999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" event={"ID":"1188575b-2c3d-4bbb-877a-11f4ddd33629","Type":"ContainerStarted","Data":"ffa89f6c16250b61b0cf7d948eb4623ad89aaa68b20b5ed9945925364052d383"} Jul 24 02:09:06.600563 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:06.600475 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62"] Jul 24 02:09:06.602970 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:09:06.602947 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc014f80f_1324_4e56_b784_dd80d163e4c9.slice/crio-2cb2caed6e2a98ed92a281a068f59373ad3a898cb3c8b3fdee1db08770d32bf6 WatchSource:0}: Error finding container 2cb2caed6e2a98ed92a281a068f59373ad3a898cb3c8b3fdee1db08770d32bf6: Status 404 returned error can't find the container with id 2cb2caed6e2a98ed92a281a068f59373ad3a898cb3c8b3fdee1db08770d32bf6 Jul 24 02:09:07.513890 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.513847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" event={"ID":"c014f80f-1324-4e56-b784-dd80d163e4c9","Type":"ContainerStarted","Data":"3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc"} Jul 24 02:09:07.514351 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.513955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" event={"ID":"c014f80f-1324-4e56-b784-dd80d163e4c9","Type":"ContainerStarted","Data":"2cb2caed6e2a98ed92a281a068f59373ad3a898cb3c8b3fdee1db08770d32bf6"} Jul 24 02:09:07.514351 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.513991 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:07.515257 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.515232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" event={"ID":"1188575b-2c3d-4bbb-877a-11f4ddd33629","Type":"ContainerStarted","Data":"f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed"} Jul 24 02:09:07.515388 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.515370 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:07.536937 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.536891 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" podStartSLOduration=1.536877177 podStartE2EDuration="1.536877177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:09:07.536504568 +0000 UTC m=+2356.001638987" watchObservedRunningTime="2026-07-24 02:09:07.536877177 +0000 UTC m=+2356.002011585" Jul 24 02:09:07.561599 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:07.561550 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" podStartSLOduration=2.561534686 podStartE2EDuration="2.561534686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:09:07.561327625 +0000 UTC m=+2356.026462062" watchObservedRunningTime="2026-07-24 02:09:07.561534686 +0000 UTC m=+2356.026669102" Jul 24 02:09:09.476329 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:09.476298 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:09:09.476708 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:09.476374 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:09:18.520978 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:18.520942 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:09:18.521461 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:18.521015 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:09:51.985914 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:51.985881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:09:51.995359 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:09:51.995336 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:12:34.779351 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.779313 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9"] Jul 24 02:12:34.782570 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.782554 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:34.785317 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.785298 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6bcabd\"/\"kube-root-ca.crt\"" Jul 24 02:12:34.786436 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.786404 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6bcabd\"/\"default-dockercfg-mkp58\"" Jul 24 02:12:34.786544 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.786453 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-6bcabd\"/\"sub-test-model-6bcabd-kserve-self-signed-certs\"" Jul 24 02:12:34.786544 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.786501 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-6bcabd\"/\"openshift-service-ca.crt\"" Jul 24 02:12:34.790563 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.790544 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9"] Jul 24 02:12:34.953837 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.953802 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-home\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:34.954028 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.953853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68vdw\" (UniqueName: \"kubernetes.io/projected/cbd997c6-f82e-4a51-8d03-79afa4a8b998-kube-api-access-68vdw\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:34.954028 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.953970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tmp-dir\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:34.954028 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.954007 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tls-certs\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:34.954199 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.954035 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-model-cache\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:34.954199 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:34.954099 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-dshm\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.054849 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.054755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tls-certs\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.054849 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.054797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-model-cache\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055049 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.054902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-dshm\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055049 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.054945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-home\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055049 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.054994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68vdw\" (UniqueName: \"kubernetes.io/projected/cbd997c6-f82e-4a51-8d03-79afa4a8b998-kube-api-access-68vdw\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055214 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.055069 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tmp-dir\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055214 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.055184 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-model-cache\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055363 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.055335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-home\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.055498 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.055366 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tmp-dir\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.057182 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.057159 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-dshm\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.057604 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.057581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tls-certs\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.062796 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.062772 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68vdw\" (UniqueName: \"kubernetes.io/projected/cbd997c6-f82e-4a51-8d03-79afa4a8b998-kube-api-access-68vdw\") pod \"sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.092699 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.092665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:35.223642 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.223617 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9"] Jul 24 02:12:35.225868 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:12:35.225842 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbd997c6_f82e_4a51_8d03_79afa4a8b998.slice/crio-2c81200175409b065eb32edb4fbf5f26badb8a4e01080f130e1e26fa4ad87453 WatchSource:0}: Error finding container 2c81200175409b065eb32edb4fbf5f26badb8a4e01080f130e1e26fa4ad87453: Status 404 returned error can't find the container with id 2c81200175409b065eb32edb4fbf5f26badb8a4e01080f130e1e26fa4ad87453 Jul 24 02:12:35.391994 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.391962 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" event={"ID":"cbd997c6-f82e-4a51-8d03-79afa4a8b998","Type":"ContainerStarted","Data":"1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000"} Jul 24 02:12:35.392170 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.392000 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" event={"ID":"cbd997c6-f82e-4a51-8d03-79afa4a8b998","Type":"ContainerStarted","Data":"2c81200175409b065eb32edb4fbf5f26badb8a4e01080f130e1e26fa4ad87453"} Jul 24 02:12:35.410317 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:35.410273 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" podStartSLOduration=1.410257608 podStartE2EDuration="1.410257608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:12:35.408937806 +0000 UTC m=+2563.874072248" watchObservedRunningTime="2026-07-24 02:12:35.410257608 +0000 UTC m=+2563.875392025" Jul 24 02:12:45.093371 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:45.093326 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:45.093371 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:45.093387 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:45.105984 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:45.105950 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:12:45.445668 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:12:45.445631 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:13:39.348459 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:39.348384 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9"] Jul 24 02:13:54.367507 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.367442 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" podUID="cbd997c6-f82e-4a51-8d03-79afa4a8b998" containerName="main" containerID="cri-o://1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000" gracePeriod=45 Jul 24 02:13:54.611981 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.611955 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:13:54.727096 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.727060 2564 generic.go:356] "Generic (PLEG): container finished" podID="cbd997c6-f82e-4a51-8d03-79afa4a8b998" containerID="1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000" exitCode=0 Jul 24 02:13:54.727294 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.727109 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" Jul 24 02:13:54.727294 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.727164 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" event={"ID":"cbd997c6-f82e-4a51-8d03-79afa4a8b998","Type":"ContainerDied","Data":"1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000"} Jul 24 02:13:54.727294 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.727209 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9" event={"ID":"cbd997c6-f82e-4a51-8d03-79afa4a8b998","Type":"ContainerDied","Data":"2c81200175409b065eb32edb4fbf5f26badb8a4e01080f130e1e26fa4ad87453"} Jul 24 02:13:54.727294 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.727230 2564 scope.go:117] "RemoveContainer" containerID="1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000" Jul 24 02:13:54.731455 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731438 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-dshm\") pod \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " Jul 24 02:13:54.731553 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731480 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-model-cache\") pod \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " Jul 24 02:13:54.731553 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731508 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68vdw\" (UniqueName: \"kubernetes.io/projected/cbd997c6-f82e-4a51-8d03-79afa4a8b998-kube-api-access-68vdw\") pod \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " Jul 24 02:13:54.731553 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731534 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tmp-dir\") pod \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " Jul 24 02:13:54.731711 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731566 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-home\") pod \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " Jul 24 02:13:54.731711 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731599 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tls-certs\") pod \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\" (UID: \"cbd997c6-f82e-4a51-8d03-79afa4a8b998\") " Jul 24 02:13:54.731816 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731750 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-model-cache" (OuterVolumeSpecName: "model-cache") pod "cbd997c6-f82e-4a51-8d03-79afa4a8b998" (UID: "cbd997c6-f82e-4a51-8d03-79afa4a8b998"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 02:13:54.731879 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731833 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cbd997c6-f82e-4a51-8d03-79afa4a8b998" (UID: "cbd997c6-f82e-4a51-8d03-79afa4a8b998"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 02:13:54.731879 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731844 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-home" (OuterVolumeSpecName: "home") pod "cbd997c6-f82e-4a51-8d03-79afa4a8b998" (UID: "cbd997c6-f82e-4a51-8d03-79afa4a8b998"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 02:13:54.731994 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.731881 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-model-cache\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:13:54.733811 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.733781 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-dshm" (OuterVolumeSpecName: "dshm") pod "cbd997c6-f82e-4a51-8d03-79afa4a8b998" (UID: "cbd997c6-f82e-4a51-8d03-79afa4a8b998"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 02:13:54.733811 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.733789 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cbd997c6-f82e-4a51-8d03-79afa4a8b998" (UID: "cbd997c6-f82e-4a51-8d03-79afa4a8b998"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 02:13:54.733976 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.733951 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbd997c6-f82e-4a51-8d03-79afa4a8b998-kube-api-access-68vdw" (OuterVolumeSpecName: "kube-api-access-68vdw") pod "cbd997c6-f82e-4a51-8d03-79afa4a8b998" (UID: "cbd997c6-f82e-4a51-8d03-79afa4a8b998"). InnerVolumeSpecName "kube-api-access-68vdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:13:54.736158 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.736120 2564 scope.go:117] "RemoveContainer" containerID="1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000" Jul 24 02:13:54.736450 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:13:54.736423 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000\": container with ID starting with 1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000 not found: ID does not exist" containerID="1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000" Jul 24 02:13:54.736520 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.736462 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000"} err="failed to get container status \"1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000\": rpc error: code = NotFound desc = could not find container \"1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000\": container with ID starting with 1ebb20f87af993271be0411d5e79119863f39d6e2ecca03631843e4a9cc8a000 not found: ID does not exist" Jul 24 02:13:54.832559 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.832522 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68vdw\" (UniqueName: \"kubernetes.io/projected/cbd997c6-f82e-4a51-8d03-79afa4a8b998-kube-api-access-68vdw\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:13:54.832559 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.832556 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tmp-dir\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:13:54.832559 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.832567 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-home\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:13:54.832789 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.832576 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbd997c6-f82e-4a51-8d03-79afa4a8b998-tls-certs\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:13:54.832789 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:54.832584 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbd997c6-f82e-4a51-8d03-79afa4a8b998-dshm\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:13:55.048567 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:55.048535 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9"] Jul 24 02:13:55.051923 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:55.051899 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-6bcabd/sub-test-model-6bcabd-kserve-5fcbd9b88c-b8rk9"] Jul 24 02:13:55.992245 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:13:55.992209 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbd997c6-f82e-4a51-8d03-79afa4a8b998" path="/var/lib/kubelet/pods/cbd997c6-f82e-4a51-8d03-79afa4a8b998/volumes" Jul 24 02:14:52.009002 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:14:52.008975 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:14:52.018509 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:14:52.018487 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:15:20.169053 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.169015 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7797-140117713073616"] Jul 24 02:15:20.169538 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.169342 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbd997c6-f82e-4a51-8d03-79afa4a8b998" containerName="main" Jul 24 02:15:20.169538 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.169354 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbd997c6-f82e-4a51-8d03-79afa4a8b998" containerName="main" Jul 24 02:15:20.169538 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.169411 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbd997c6-f82e-4a51-8d03-79afa4a8b998" containerName="main" Jul 24 02:15:20.172437 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.172421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:20.174837 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.174815 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q8hnt\"" Jul 24 02:15:20.179792 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.179768 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7797-140117713073616"] Jul 24 02:15:20.259946 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.259911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsct4\" (UniqueName: \"kubernetes.io/projected/4c21bcf5-1f73-4a2c-91d3-664f96379110-kube-api-access-fsct4\") pod \"test-curl-7797-140117713073616\" (UID: \"4c21bcf5-1f73-4a2c-91d3-664f96379110\") " pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:20.361442 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.361412 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsct4\" (UniqueName: \"kubernetes.io/projected/4c21bcf5-1f73-4a2c-91d3-664f96379110-kube-api-access-fsct4\") pod \"test-curl-7797-140117713073616\" (UID: \"4c21bcf5-1f73-4a2c-91d3-664f96379110\") " pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:20.369121 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.369101 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsct4\" (UniqueName: \"kubernetes.io/projected/4c21bcf5-1f73-4a2c-91d3-664f96379110-kube-api-access-fsct4\") pod \"test-curl-7797-140117713073616\" (UID: \"4c21bcf5-1f73-4a2c-91d3-664f96379110\") " pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:20.482732 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.482656 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:20.613925 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.613892 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7797-140117713073616"] Jul 24 02:15:20.616351 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:15:20.616321 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c21bcf5_1f73_4a2c_91d3_664f96379110.slice/crio-bff2aea5455804632e309303ef1ed8fa5c78404409f9942f9b6359f21fe46c8d WatchSource:0}: Error finding container bff2aea5455804632e309303ef1ed8fa5c78404409f9942f9b6359f21fe46c8d: Status 404 returned error can't find the container with id bff2aea5455804632e309303ef1ed8fa5c78404409f9942f9b6359f21fe46c8d Jul 24 02:15:20.617781 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:20.617760 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 02:15:21.086457 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:21.086420 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7797-140117713073616" event={"ID":"4c21bcf5-1f73-4a2c-91d3-664f96379110","Type":"ContainerStarted","Data":"bff2aea5455804632e309303ef1ed8fa5c78404409f9942f9b6359f21fe46c8d"} Jul 24 02:15:22.092179 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:22.092117 2564 generic.go:356] "Generic (PLEG): container finished" podID="4c21bcf5-1f73-4a2c-91d3-664f96379110" containerID="19d650a3dbb0267f9aff5485754ddb2e32134c787bc75b1b1a5f8229699a2890" exitCode=0 Jul 24 02:15:22.092179 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:22.092163 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7797-140117713073616" event={"ID":"4c21bcf5-1f73-4a2c-91d3-664f96379110","Type":"ContainerDied","Data":"19d650a3dbb0267f9aff5485754ddb2e32134c787bc75b1b1a5f8229699a2890"} Jul 24 02:15:23.227116 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.227091 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:23.261740 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.261709 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7797-140117713073616_4c21bcf5-1f73-4a2c-91d3-664f96379110/test-curl-7797-140117713073616/0.log" Jul 24 02:15:23.280996 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.280973 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsct4\" (UniqueName: \"kubernetes.io/projected/4c21bcf5-1f73-4a2c-91d3-664f96379110-kube-api-access-fsct4\") pod \"4c21bcf5-1f73-4a2c-91d3-664f96379110\" (UID: \"4c21bcf5-1f73-4a2c-91d3-664f96379110\") " Jul 24 02:15:23.283095 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.283067 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c21bcf5-1f73-4a2c-91d3-664f96379110-kube-api-access-fsct4" (OuterVolumeSpecName: "kube-api-access-fsct4") pod "4c21bcf5-1f73-4a2c-91d3-664f96379110" (UID: "4c21bcf5-1f73-4a2c-91d3-664f96379110"). InnerVolumeSpecName "kube-api-access-fsct4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:15:23.285899 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.285875 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7797-140117713073616"] Jul 24 02:15:23.289169 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.289152 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7797-140117713073616"] Jul 24 02:15:23.381596 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.381567 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsct4\" (UniqueName: \"kubernetes.io/projected/4c21bcf5-1f73-4a2c-91d3-664f96379110-kube-api-access-fsct4\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:15:23.992177 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:23.992120 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c21bcf5-1f73-4a2c-91d3-664f96379110" path="/var/lib/kubelet/pods/4c21bcf5-1f73-4a2c-91d3-664f96379110/volumes" Jul 24 02:15:24.101836 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:24.101805 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7797-140117713073616" Jul 24 02:15:24.101836 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:24.101826 2564 scope.go:117] "RemoveContainer" containerID="19d650a3dbb0267f9aff5485754ddb2e32134c787bc75b1b1a5f8229699a2890" Jul 24 02:15:31.888929 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.888895 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l"] Jul 24 02:15:31.889305 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.889232 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c21bcf5-1f73-4a2c-91d3-664f96379110" containerName="test-curl-7797-140117713073616" Jul 24 02:15:31.889305 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.889245 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c21bcf5-1f73-4a2c-91d3-664f96379110" containerName="test-curl-7797-140117713073616" Jul 24 02:15:31.889305 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.889303 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c21bcf5-1f73-4a2c-91d3-664f96379110" containerName="test-curl-7797-140117713073616" Jul 24 02:15:31.892177 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.892160 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:31.895736 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.895710 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-a002f7-dockercfg-kkmcg\"" Jul 24 02:15:31.895882 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.895756 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-a002f7\"" Jul 24 02:15:31.906895 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.906873 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l"] Jul 24 02:15:31.955654 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.955624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz2s7\" (UniqueName: \"kubernetes.io/projected/06ac05d7-abfd-4860-8210-8af316f08358-kube-api-access-wz2s7\") pod \"payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:31.955807 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:31.955687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ac05d7-abfd-4860-8210-8af316f08358-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:32.056447 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:32.056411 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ac05d7-abfd-4860-8210-8af316f08358-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:32.056635 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:32.056514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz2s7\" (UniqueName: \"kubernetes.io/projected/06ac05d7-abfd-4860-8210-8af316f08358-kube-api-access-wz2s7\") pod \"payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:32.057087 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:32.057066 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ac05d7-abfd-4860-8210-8af316f08358-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:32.065803 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:32.065775 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz2s7\" (UniqueName: \"kubernetes.io/projected/06ac05d7-abfd-4860-8210-8af316f08358-kube-api-access-wz2s7\") pod \"payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:32.202792 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:32.202707 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:32.330246 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:32.330196 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l"] Jul 24 02:15:32.332968 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:15:32.332937 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ac05d7_abfd_4860_8210_8af316f08358.slice/crio-ccdfc2d02b5ed74d7e4b6cabc219c3a97ba0b11e7400503e01583b0b1e2f1b3b WatchSource:0}: Error finding container ccdfc2d02b5ed74d7e4b6cabc219c3a97ba0b11e7400503e01583b0b1e2f1b3b: Status 404 returned error can't find the container with id ccdfc2d02b5ed74d7e4b6cabc219c3a97ba0b11e7400503e01583b0b1e2f1b3b Jul 24 02:15:33.144540 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:33.144506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" event={"ID":"06ac05d7-abfd-4860-8210-8af316f08358","Type":"ContainerStarted","Data":"1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287"} Jul 24 02:15:33.144909 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:33.144558 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" event={"ID":"06ac05d7-abfd-4860-8210-8af316f08358","Type":"ContainerStarted","Data":"ccdfc2d02b5ed74d7e4b6cabc219c3a97ba0b11e7400503e01583b0b1e2f1b3b"} Jul 24 02:15:33.144909 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:33.144620 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:33.161522 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:33.161468 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" podStartSLOduration=2.161450846 podStartE2EDuration="2.161450846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:15:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:15:33.160204329 +0000 UTC m=+2741.625338765" watchObservedRunningTime="2026-07-24 02:15:33.161450846 +0000 UTC m=+2741.626585264" Jul 24 02:15:44.149988 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:44.149957 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:15:51.543217 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.543177 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb"] Jul 24 02:15:51.546688 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.546665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.549620 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.549594 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-477d74\"" Jul 24 02:15:51.549727 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.549702 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-477d74-dockercfg-792km\"" Jul 24 02:15:51.556634 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.556611 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb"] Jul 24 02:15:51.573009 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.572987 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb"] Jul 24 02:15:51.576097 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.576080 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.587170 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.587144 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb"] Jul 24 02:15:51.621798 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.621768 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61215352-807a-4a0d-9897-68dd71f200ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.621950 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.621801 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frlwl\" (UniqueName: \"kubernetes.io/projected/61215352-807a-4a0d-9897-68dd71f200ee-kube-api-access-frlwl\") pod \"payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.722758 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.722710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ksl7\" (UniqueName: \"kubernetes.io/projected/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-kube-api-access-5ksl7\") pod \"payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.722942 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.722799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61215352-807a-4a0d-9897-68dd71f200ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.722942 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.722819 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frlwl\" (UniqueName: \"kubernetes.io/projected/61215352-807a-4a0d-9897-68dd71f200ee-kube-api-access-frlwl\") pod \"payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.722942 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.722847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.723548 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.723523 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61215352-807a-4a0d-9897-68dd71f200ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.731782 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.731760 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frlwl\" (UniqueName: \"kubernetes.io/projected/61215352-807a-4a0d-9897-68dd71f200ee-kube-api-access-frlwl\") pod \"payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.823431 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.823334 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.823431 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.823383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ksl7\" (UniqueName: \"kubernetes.io/projected/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-kube-api-access-5ksl7\") pod \"payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.823990 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.823967 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.831871 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.831847 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ksl7\" (UniqueName: \"kubernetes.io/projected/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-kube-api-access-5ksl7\") pod \"payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:51.858819 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.858800 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-477d74-dockercfg-792km\"" Jul 24 02:15:51.867163 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.867143 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:51.885953 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:51.885933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:52.212644 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:52.212619 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb"] Jul 24 02:15:52.215023 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:15:52.214998 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61215352_807a_4a0d_9897_68dd71f200ee.slice/crio-be19970ae515e88915a514c26b337a2c6da5dbf01beff5c99ce1456be3a57feb WatchSource:0}: Error finding container be19970ae515e88915a514c26b337a2c6da5dbf01beff5c99ce1456be3a57feb: Status 404 returned error can't find the container with id be19970ae515e88915a514c26b337a2c6da5dbf01beff5c99ce1456be3a57feb Jul 24 02:15:52.224757 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:52.224729 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" event={"ID":"61215352-807a-4a0d-9897-68dd71f200ee","Type":"ContainerStarted","Data":"be19970ae515e88915a514c26b337a2c6da5dbf01beff5c99ce1456be3a57feb"} Jul 24 02:15:52.241274 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:52.241252 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb"] Jul 24 02:15:52.251114 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:15:52.251083 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2291e112_d90a_4b4e_82de_2d3ef1cde0ee.slice/crio-71f7a291a64708aada332f34f832bb04911a30f264e0ab24d0aecf78abac16a7 WatchSource:0}: Error finding container 71f7a291a64708aada332f34f832bb04911a30f264e0ab24d0aecf78abac16a7: Status 404 returned error can't find the container with id 71f7a291a64708aada332f34f832bb04911a30f264e0ab24d0aecf78abac16a7 Jul 24 02:15:53.230087 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.230049 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" event={"ID":"61215352-807a-4a0d-9897-68dd71f200ee","Type":"ContainerStarted","Data":"b92908446bb73afdc189445d5c1810450b7deba62ea6f8aee0861a1b822807bf"} Jul 24 02:15:53.230592 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.230119 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:15:53.231541 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.231508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" event={"ID":"2291e112-d90a-4b4e-82de-2d3ef1cde0ee","Type":"ContainerStarted","Data":"8aed9de6da34afb62d55b52ab75bb05ab831fdc351bf895ade418c24bf214c2f"} Jul 24 02:15:53.231648 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.231560 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" event={"ID":"2291e112-d90a-4b4e-82de-2d3ef1cde0ee","Type":"ContainerStarted","Data":"71f7a291a64708aada332f34f832bb04911a30f264e0ab24d0aecf78abac16a7"} Jul 24 02:15:53.231648 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.231583 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:15:53.247859 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.247815 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" podStartSLOduration=2.247799878 podStartE2EDuration="2.247799878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:15:53.246399433 +0000 UTC m=+2761.711533849" watchObservedRunningTime="2026-07-24 02:15:53.247799878 +0000 UTC m=+2761.712934295" Jul 24 02:15:53.263021 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:15:53.262968 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" podStartSLOduration=2.262950829 podStartE2EDuration="2.262950829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:15:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:15:53.261504035 +0000 UTC m=+2761.726638451" watchObservedRunningTime="2026-07-24 02:15:53.262950829 +0000 UTC m=+2761.728085248" Jul 24 02:16:04.237774 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:16:04.237742 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:16:04.238189 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:16:04.237811 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:16:16.093553 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:16:16.093460 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb_2291e112-d90a-4b4e-82de-2d3ef1cde0ee/payload-processing/0.log" Jul 24 02:16:57.732934 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:16:57.732902 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l_06ac05d7-abfd-4860-8210-8af316f08358/payload-processing/0.log" Jul 24 02:16:57.993669 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:16:57.993594 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb_2291e112-d90a-4b4e-82de-2d3ef1cde0ee/payload-processing/0.log" Jul 24 02:17:04.914984 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.914952 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm"] Jul 24 02:17:04.918490 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.918469 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:04.920999 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.920982 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e50c2c\"" Jul 24 02:17:04.921065 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.920991 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e50c2c-dockercfg-jj8tx\"" Jul 24 02:17:04.928672 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.928649 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm"] Jul 24 02:17:04.950059 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.950029 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc"] Jul 24 02:17:04.952714 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.952691 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:04.952813 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.952759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cvns\" (UniqueName: \"kubernetes.io/projected/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-kube-api-access-4cvns\") pod \"payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:04.953882 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.953863 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:04.961843 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:04.961820 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc"] Jul 24 02:17:05.054091 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.054048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cvns\" (UniqueName: \"kubernetes.io/projected/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-kube-api-access-4cvns\") pod \"payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:05.054324 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.054149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:05.054907 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.054885 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:05.062335 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.062308 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cvns\" (UniqueName: \"kubernetes.io/projected/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-kube-api-access-4cvns\") pod \"payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:05.155237 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.155192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnfsx\" (UniqueName: \"kubernetes.io/projected/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-kube-api-access-jnfsx\") pod \"payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.155425 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.155282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.229276 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.229172 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:05.255780 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.255747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnfsx\" (UniqueName: \"kubernetes.io/projected/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-kube-api-access-jnfsx\") pod \"payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.255939 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.255799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.256482 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.256460 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.264649 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.264623 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnfsx\" (UniqueName: \"kubernetes.io/projected/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-kube-api-access-jnfsx\") pod \"payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.359535 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.359513 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm"] Jul 24 02:17:05.361308 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:17:05.361282 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128e71de_0a63_48b1_a8cf_e91edf2a9bdf.slice/crio-c94f57f527036a367c5f94cee048c0f127647478996585a0d71d708128851c4c WatchSource:0}: Error finding container c94f57f527036a367c5f94cee048c0f127647478996585a0d71d708128851c4c: Status 404 returned error can't find the container with id c94f57f527036a367c5f94cee048c0f127647478996585a0d71d708128851c4c Jul 24 02:17:05.541868 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.541775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" event={"ID":"128e71de-0a63-48b1-a8cf-e91edf2a9bdf","Type":"ContainerStarted","Data":"f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370"} Jul 24 02:17:05.541868 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.541836 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" event={"ID":"128e71de-0a63-48b1-a8cf-e91edf2a9bdf","Type":"ContainerStarted","Data":"c94f57f527036a367c5f94cee048c0f127647478996585a0d71d708128851c4c"} Jul 24 02:17:05.542062 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.541911 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:05.559438 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.559389 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" podStartSLOduration=1.559371975 podStartE2EDuration="1.559371975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:17:05.558671335 +0000 UTC m=+2834.023805765" watchObservedRunningTime="2026-07-24 02:17:05.559371975 +0000 UTC m=+2834.024506391" Jul 24 02:17:05.564789 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.564765 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:05.898247 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:05.898224 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc"] Jul 24 02:17:05.900163 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:17:05.900135 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed0c564_ad6b_4c4b_a3c8_5892c7cb1edc.slice/crio-d4234211723990e394d692abc7c95ce560d57884d985049f04aa9ad1c9707307 WatchSource:0}: Error finding container d4234211723990e394d692abc7c95ce560d57884d985049f04aa9ad1c9707307: Status 404 returned error can't find the container with id d4234211723990e394d692abc7c95ce560d57884d985049f04aa9ad1c9707307 Jul 24 02:17:06.550750 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:06.550706 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" event={"ID":"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc","Type":"ContainerStarted","Data":"496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5"} Jul 24 02:17:06.551198 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:06.550772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" event={"ID":"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc","Type":"ContainerStarted","Data":"d4234211723990e394d692abc7c95ce560d57884d985049f04aa9ad1c9707307"} Jul 24 02:17:06.568266 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:06.568209 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" podStartSLOduration=2.568190496 podStartE2EDuration="2.568190496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 02:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:17:06.565943718 +0000 UTC m=+2835.031078136" watchObservedRunningTime="2026-07-24 02:17:06.568190496 +0000 UTC m=+2835.033324914" Jul 24 02:17:07.555794 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:07.555756 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:16.552705 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:16.552670 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:18.562485 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:18.562454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:35.318028 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.317993 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc"] Jul 24 02:17:35.320451 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.318279 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" podUID="fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" containerName="payload-processing" containerID="cri-o://496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5" gracePeriod=30 Jul 24 02:17:35.329574 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.329544 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm"] Jul 24 02:17:35.329824 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.329779 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" podUID="128e71de-0a63-48b1-a8cf-e91edf2a9bdf" containerName="payload-pre-processing" containerID="cri-o://f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370" gracePeriod=30 Jul 24 02:17:35.581972 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.581914 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:35.611463 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.611438 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:35.683492 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.683462 2564 generic.go:356] "Generic (PLEG): container finished" podID="fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" containerID="496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5" exitCode=0 Jul 24 02:17:35.683675 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.683506 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" Jul 24 02:17:35.683675 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.683546 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" event={"ID":"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc","Type":"ContainerDied","Data":"496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5"} Jul 24 02:17:35.683675 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.683600 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc" event={"ID":"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc","Type":"ContainerDied","Data":"d4234211723990e394d692abc7c95ce560d57884d985049f04aa9ad1c9707307"} Jul 24 02:17:35.683675 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.683627 2564 scope.go:117] "RemoveContainer" containerID="496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5" Jul 24 02:17:35.684783 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.684706 2564 generic.go:356] "Generic (PLEG): container finished" podID="128e71de-0a63-48b1-a8cf-e91edf2a9bdf" containerID="f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370" exitCode=0 Jul 24 02:17:35.684783 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.684747 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" Jul 24 02:17:35.684783 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.684760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" event={"ID":"128e71de-0a63-48b1-a8cf-e91edf2a9bdf","Type":"ContainerDied","Data":"f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370"} Jul 24 02:17:35.684903 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.684797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm" event={"ID":"128e71de-0a63-48b1-a8cf-e91edf2a9bdf","Type":"ContainerDied","Data":"c94f57f527036a367c5f94cee048c0f127647478996585a0d71d708128851c4c"} Jul 24 02:17:35.691891 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.691872 2564 scope.go:117] "RemoveContainer" containerID="496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5" Jul 24 02:17:35.692159 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:17:35.692115 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5\": container with ID starting with 496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5 not found: ID does not exist" containerID="496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5" Jul 24 02:17:35.692256 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.692167 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5"} err="failed to get container status \"496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5\": rpc error: code = NotFound desc = could not find container \"496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5\": container with ID starting with 496dec3ea61895e2a9a659912f8738113475a6363dbe6f2769c373394b80f4b5 not found: ID does not exist" Jul 24 02:17:35.692256 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.692183 2564 scope.go:117] "RemoveContainer" containerID="f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370" Jul 24 02:17:35.699627 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.699610 2564 scope.go:117] "RemoveContainer" containerID="f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370" Jul 24 02:17:35.699868 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:17:35.699851 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370\": container with ID starting with f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370 not found: ID does not exist" containerID="f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370" Jul 24 02:17:35.699915 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.699873 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370"} err="failed to get container status \"f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370\": rpc error: code = NotFound desc = could not find container \"f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370\": container with ID starting with f90f99c6c6ed6b3bc695945f6f545b296f0917b686928aff776d95c106d94370 not found: ID does not exist" Jul 24 02:17:35.713191 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713172 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnfsx\" (UniqueName: \"kubernetes.io/projected/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-kube-api-access-jnfsx\") pod \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " Jul 24 02:17:35.713259 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713216 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cvns\" (UniqueName: \"kubernetes.io/projected/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-kube-api-access-4cvns\") pod \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " Jul 24 02:17:35.713259 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713240 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-plugins-config-volume\") pod \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\" (UID: \"128e71de-0a63-48b1-a8cf-e91edf2a9bdf\") " Jul 24 02:17:35.713498 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713479 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-plugins-config-volume\") pod \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\" (UID: \"fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc\") " Jul 24 02:17:35.713566 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713546 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "128e71de-0a63-48b1-a8cf-e91edf2a9bdf" (UID: "128e71de-0a63-48b1-a8cf-e91edf2a9bdf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:17:35.713769 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713752 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:17:35.713840 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.713798 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" (UID: "fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:17:35.715183 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.715164 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-kube-api-access-4cvns" (OuterVolumeSpecName: "kube-api-access-4cvns") pod "128e71de-0a63-48b1-a8cf-e91edf2a9bdf" (UID: "128e71de-0a63-48b1-a8cf-e91edf2a9bdf"). InnerVolumeSpecName "kube-api-access-4cvns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:17:35.715264 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.715233 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-kube-api-access-jnfsx" (OuterVolumeSpecName: "kube-api-access-jnfsx") pod "fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" (UID: "fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc"). InnerVolumeSpecName "kube-api-access-jnfsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:17:35.814612 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.814574 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnfsx\" (UniqueName: \"kubernetes.io/projected/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-kube-api-access-jnfsx\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:17:35.814612 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.814604 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cvns\" (UniqueName: \"kubernetes.io/projected/128e71de-0a63-48b1-a8cf-e91edf2a9bdf-kube-api-access-4cvns\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:17:35.814612 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:35.814615 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:17:36.013903 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:36.013866 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc"] Jul 24 02:17:36.020009 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:36.019980 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e50c2c-669589bc6-2kwtc"] Jul 24 02:17:36.034510 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:36.034485 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm"] Jul 24 02:17:36.042592 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:36.042564 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e50c2c-798f88864-2mpjm"] Jul 24 02:17:37.992992 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:37.992958 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="128e71de-0a63-48b1-a8cf-e91edf2a9bdf" path="/var/lib/kubelet/pods/128e71de-0a63-48b1-a8cf-e91edf2a9bdf/volumes" Jul 24 02:17:37.993391 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:17:37.993339 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" path="/var/lib/kubelet/pods/fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc/volumes" Jul 24 02:18:08.482878 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.482839 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l"] Jul 24 02:18:08.483390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.483061 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" podUID="06ac05d7-abfd-4860-8210-8af316f08358" containerName="payload-processing" containerID="cri-o://1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287" gracePeriod=30 Jul 24 02:18:08.733789 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.733734 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:18:08.803609 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.803575 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz2s7\" (UniqueName: \"kubernetes.io/projected/06ac05d7-abfd-4860-8210-8af316f08358-kube-api-access-wz2s7\") pod \"06ac05d7-abfd-4860-8210-8af316f08358\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " Jul 24 02:18:08.803813 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.803696 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ac05d7-abfd-4860-8210-8af316f08358-plugins-config-volume\") pod \"06ac05d7-abfd-4860-8210-8af316f08358\" (UID: \"06ac05d7-abfd-4860-8210-8af316f08358\") " Jul 24 02:18:08.804030 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.804003 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06ac05d7-abfd-4860-8210-8af316f08358-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06ac05d7-abfd-4860-8210-8af316f08358" (UID: "06ac05d7-abfd-4860-8210-8af316f08358"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:18:08.805698 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.805675 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ac05d7-abfd-4860-8210-8af316f08358-kube-api-access-wz2s7" (OuterVolumeSpecName: "kube-api-access-wz2s7") pod "06ac05d7-abfd-4860-8210-8af316f08358" (UID: "06ac05d7-abfd-4860-8210-8af316f08358"). InnerVolumeSpecName "kube-api-access-wz2s7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:18:08.835710 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.835682 2564 generic.go:356] "Generic (PLEG): container finished" podID="06ac05d7-abfd-4860-8210-8af316f08358" containerID="1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287" exitCode=0 Jul 24 02:18:08.835844 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.835714 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" event={"ID":"06ac05d7-abfd-4860-8210-8af316f08358","Type":"ContainerDied","Data":"1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287"} Jul 24 02:18:08.835844 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.835740 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" Jul 24 02:18:08.835844 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.835782 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l" event={"ID":"06ac05d7-abfd-4860-8210-8af316f08358","Type":"ContainerDied","Data":"ccdfc2d02b5ed74d7e4b6cabc219c3a97ba0b11e7400503e01583b0b1e2f1b3b"} Jul 24 02:18:08.835844 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.835815 2564 scope.go:117] "RemoveContainer" containerID="1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287" Jul 24 02:18:08.844149 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.844113 2564 scope.go:117] "RemoveContainer" containerID="1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287" Jul 24 02:18:08.844423 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:18:08.844401 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287\": container with ID starting with 1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287 not found: ID does not exist" containerID="1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287" Jul 24 02:18:08.844517 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.844430 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287"} err="failed to get container status \"1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287\": rpc error: code = NotFound desc = could not find container \"1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287\": container with ID starting with 1f0527b3266381b5a55830db026c6aba6197f6655a6e8b8f0e9d443c922d7287 not found: ID does not exist" Jul 24 02:18:08.860401 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.860368 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l"] Jul 24 02:18:08.863840 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.863820 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-a002f7-6865dd9bbb-5jv7l"] Jul 24 02:18:08.904887 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.904864 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ac05d7-abfd-4860-8210-8af316f08358-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:18:08.904887 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:08.904886 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz2s7\" (UniqueName: \"kubernetes.io/projected/06ac05d7-abfd-4860-8210-8af316f08358-kube-api-access-wz2s7\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:18:09.991971 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:18:09.991936 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06ac05d7-abfd-4860-8210-8af316f08358" path="/var/lib/kubelet/pods/06ac05d7-abfd-4860-8210-8af316f08358/volumes" Jul 24 02:19:13.865362 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:13.865323 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb"] Jul 24 02:19:13.865921 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:13.865612 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" podUID="2291e112-d90a-4b4e-82de-2d3ef1cde0ee" containerName="payload-processing" containerID="cri-o://8aed9de6da34afb62d55b52ab75bb05ab831fdc351bf895ade418c24bf214c2f" gracePeriod=30 Jul 24 02:19:13.872782 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:13.872758 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb"] Jul 24 02:19:13.873079 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:13.873033 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" podUID="61215352-807a-4a0d-9897-68dd71f200ee" containerName="payload-pre-processing" containerID="cri-o://b92908446bb73afdc189445d5c1810450b7deba62ea6f8aee0861a1b822807bf" gracePeriod=30 Jul 24 02:19:14.121028 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.120999 2564 generic.go:356] "Generic (PLEG): container finished" podID="61215352-807a-4a0d-9897-68dd71f200ee" containerID="b92908446bb73afdc189445d5c1810450b7deba62ea6f8aee0861a1b822807bf" exitCode=0 Jul 24 02:19:14.121180 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.121065 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" event={"ID":"61215352-807a-4a0d-9897-68dd71f200ee","Type":"ContainerDied","Data":"b92908446bb73afdc189445d5c1810450b7deba62ea6f8aee0861a1b822807bf"} Jul 24 02:19:14.122322 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.122300 2564 generic.go:356] "Generic (PLEG): container finished" podID="2291e112-d90a-4b4e-82de-2d3ef1cde0ee" containerID="8aed9de6da34afb62d55b52ab75bb05ab831fdc351bf895ade418c24bf214c2f" exitCode=0 Jul 24 02:19:14.122437 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.122367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" event={"ID":"2291e112-d90a-4b4e-82de-2d3ef1cde0ee","Type":"ContainerDied","Data":"8aed9de6da34afb62d55b52ab75bb05ab831fdc351bf895ade418c24bf214c2f"} Jul 24 02:19:14.146325 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.146308 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:19:14.149184 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.149168 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:19:14.220047 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.220010 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61215352-807a-4a0d-9897-68dd71f200ee-plugins-config-volume\") pod \"61215352-807a-4a0d-9897-68dd71f200ee\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " Jul 24 02:19:14.220047 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.220051 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-plugins-config-volume\") pod \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " Jul 24 02:19:14.220305 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.220078 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frlwl\" (UniqueName: \"kubernetes.io/projected/61215352-807a-4a0d-9897-68dd71f200ee-kube-api-access-frlwl\") pod \"61215352-807a-4a0d-9897-68dd71f200ee\" (UID: \"61215352-807a-4a0d-9897-68dd71f200ee\") " Jul 24 02:19:14.220305 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.220099 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ksl7\" (UniqueName: \"kubernetes.io/projected/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-kube-api-access-5ksl7\") pod \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\" (UID: \"2291e112-d90a-4b4e-82de-2d3ef1cde0ee\") " Jul 24 02:19:14.220487 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.220463 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2291e112-d90a-4b4e-82de-2d3ef1cde0ee" (UID: "2291e112-d90a-4b4e-82de-2d3ef1cde0ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:19:14.220554 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.220522 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61215352-807a-4a0d-9897-68dd71f200ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61215352-807a-4a0d-9897-68dd71f200ee" (UID: "61215352-807a-4a0d-9897-68dd71f200ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:19:14.222194 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.222167 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-kube-api-access-5ksl7" (OuterVolumeSpecName: "kube-api-access-5ksl7") pod "2291e112-d90a-4b4e-82de-2d3ef1cde0ee" (UID: "2291e112-d90a-4b4e-82de-2d3ef1cde0ee"). InnerVolumeSpecName "kube-api-access-5ksl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:19:14.222282 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.222263 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61215352-807a-4a0d-9897-68dd71f200ee-kube-api-access-frlwl" (OuterVolumeSpecName: "kube-api-access-frlwl") pod "61215352-807a-4a0d-9897-68dd71f200ee" (UID: "61215352-807a-4a0d-9897-68dd71f200ee"). InnerVolumeSpecName "kube-api-access-frlwl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:19:14.321199 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.321164 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61215352-807a-4a0d-9897-68dd71f200ee-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:14.321199 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.321191 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:14.321199 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.321203 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frlwl\" (UniqueName: \"kubernetes.io/projected/61215352-807a-4a0d-9897-68dd71f200ee-kube-api-access-frlwl\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:14.321412 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:14.321212 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ksl7\" (UniqueName: \"kubernetes.io/projected/2291e112-d90a-4b4e-82de-2d3ef1cde0ee-kube-api-access-5ksl7\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:15.127930 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.127896 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" Jul 24 02:19:15.128408 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.127932 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb" event={"ID":"61215352-807a-4a0d-9897-68dd71f200ee","Type":"ContainerDied","Data":"be19970ae515e88915a514c26b337a2c6da5dbf01beff5c99ce1456be3a57feb"} Jul 24 02:19:15.128408 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.128003 2564 scope.go:117] "RemoveContainer" containerID="b92908446bb73afdc189445d5c1810450b7deba62ea6f8aee0861a1b822807bf" Jul 24 02:19:15.129332 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.129270 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" event={"ID":"2291e112-d90a-4b4e-82de-2d3ef1cde0ee","Type":"ContainerDied","Data":"71f7a291a64708aada332f34f832bb04911a30f264e0ab24d0aecf78abac16a7"} Jul 24 02:19:15.129332 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.129299 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb" Jul 24 02:19:15.137517 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.137494 2564 scope.go:117] "RemoveContainer" containerID="8aed9de6da34afb62d55b52ab75bb05ab831fdc351bf895ade418c24bf214c2f" Jul 24 02:19:15.156775 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.156754 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb"] Jul 24 02:19:15.161017 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.160997 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-477d74-56668fcd84-w46sb"] Jul 24 02:19:15.172119 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.172089 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb"] Jul 24 02:19:15.175716 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.175695 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-477d74-7d89f4f99d-smvkb"] Jul 24 02:19:15.992655 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.992620 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2291e112-d90a-4b4e-82de-2d3ef1cde0ee" path="/var/lib/kubelet/pods/2291e112-d90a-4b4e-82de-2d3ef1cde0ee/volumes" Jul 24 02:19:15.993019 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:15.993004 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61215352-807a-4a0d-9897-68dd71f200ee" path="/var/lib/kubelet/pods/61215352-807a-4a0d-9897-68dd71f200ee/volumes" Jul 24 02:19:35.108571 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.108533 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr"] Jul 24 02:19:35.109053 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.108804 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" podUID="7f978d28-fd7e-45a6-98dd-d3c575dcd58e" containerName="payload-processing" containerID="cri-o://82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c" gracePeriod=30 Jul 24 02:19:35.116662 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.116632 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz"] Jul 24 02:19:35.116877 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.116845 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" podUID="865b48f8-ce23-4b68-a33c-d8cbb0fdffda" containerName="payload-pre-processing" containerID="cri-o://bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243" gracePeriod=30 Jul 24 02:19:35.384367 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.384341 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:19:35.387071 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.387051 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:19:35.500263 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.500221 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-plugins-config-volume\") pod \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " Jul 24 02:19:35.500459 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.500316 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mlmx\" (UniqueName: \"kubernetes.io/projected/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-kube-api-access-5mlmx\") pod \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " Jul 24 02:19:35.500459 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.500347 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-plugins-config-volume\") pod \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\" (UID: \"7f978d28-fd7e-45a6-98dd-d3c575dcd58e\") " Jul 24 02:19:35.500459 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.500409 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2789j\" (UniqueName: \"kubernetes.io/projected/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-kube-api-access-2789j\") pod \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\" (UID: \"865b48f8-ce23-4b68-a33c-d8cbb0fdffda\") " Jul 24 02:19:35.500713 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.500687 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "865b48f8-ce23-4b68-a33c-d8cbb0fdffda" (UID: "865b48f8-ce23-4b68-a33c-d8cbb0fdffda"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:19:35.500816 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.500789 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f978d28-fd7e-45a6-98dd-d3c575dcd58e" (UID: "7f978d28-fd7e-45a6-98dd-d3c575dcd58e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:19:35.502429 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.502406 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-kube-api-access-5mlmx" (OuterVolumeSpecName: "kube-api-access-5mlmx") pod "7f978d28-fd7e-45a6-98dd-d3c575dcd58e" (UID: "7f978d28-fd7e-45a6-98dd-d3c575dcd58e"). InnerVolumeSpecName "kube-api-access-5mlmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:19:35.502520 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.502452 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-kube-api-access-2789j" (OuterVolumeSpecName: "kube-api-access-2789j") pod "865b48f8-ce23-4b68-a33c-d8cbb0fdffda" (UID: "865b48f8-ce23-4b68-a33c-d8cbb0fdffda"). InnerVolumeSpecName "kube-api-access-2789j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:19:35.601238 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.601192 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2789j\" (UniqueName: \"kubernetes.io/projected/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-kube-api-access-2789j\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:35.601238 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.601231 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/865b48f8-ce23-4b68-a33c-d8cbb0fdffda-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:35.601238 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.601241 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mlmx\" (UniqueName: \"kubernetes.io/projected/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-kube-api-access-5mlmx\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:35.601238 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:35.601252 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f978d28-fd7e-45a6-98dd-d3c575dcd58e-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:36.223853 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.223821 2564 generic.go:356] "Generic (PLEG): container finished" podID="7f978d28-fd7e-45a6-98dd-d3c575dcd58e" containerID="82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c" exitCode=0 Jul 24 02:19:36.224364 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.223864 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" Jul 24 02:19:36.224364 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.223919 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" event={"ID":"7f978d28-fd7e-45a6-98dd-d3c575dcd58e","Type":"ContainerDied","Data":"82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c"} Jul 24 02:19:36.224364 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.223981 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr" event={"ID":"7f978d28-fd7e-45a6-98dd-d3c575dcd58e","Type":"ContainerDied","Data":"fcdb7c456f52cab0dfe5f90b61115710bc71126857f6fb6262397c0d0615c07c"} Jul 24 02:19:36.224364 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.224016 2564 scope.go:117] "RemoveContainer" containerID="82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c" Jul 24 02:19:36.225359 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.225278 2564 generic.go:356] "Generic (PLEG): container finished" podID="865b48f8-ce23-4b68-a33c-d8cbb0fdffda" containerID="bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243" exitCode=0 Jul 24 02:19:36.225359 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.225316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" event={"ID":"865b48f8-ce23-4b68-a33c-d8cbb0fdffda","Type":"ContainerDied","Data":"bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243"} Jul 24 02:19:36.225458 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.225371 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" event={"ID":"865b48f8-ce23-4b68-a33c-d8cbb0fdffda","Type":"ContainerDied","Data":"1ccf4ffa5965c6d61805d8ab039a1dba00d165595437bb0f2d780f7b4fdc5aa8"} Jul 24 02:19:36.225458 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.225326 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz" Jul 24 02:19:36.232437 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.232421 2564 scope.go:117] "RemoveContainer" containerID="82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c" Jul 24 02:19:36.232688 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:19:36.232671 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c\": container with ID starting with 82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c not found: ID does not exist" containerID="82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c" Jul 24 02:19:36.232749 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.232695 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c"} err="failed to get container status \"82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c\": rpc error: code = NotFound desc = could not find container \"82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c\": container with ID starting with 82013412d58fc60d4a66f2eeeacec854f4f90dbe15b9d38d1eb9464b29f5a43c not found: ID does not exist" Jul 24 02:19:36.232749 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.232711 2564 scope.go:117] "RemoveContainer" containerID="bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243" Jul 24 02:19:36.239850 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.239834 2564 scope.go:117] "RemoveContainer" containerID="bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243" Jul 24 02:19:36.240116 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:19:36.240097 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243\": container with ID starting with bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243 not found: ID does not exist" containerID="bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243" Jul 24 02:19:36.240204 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.240145 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243"} err="failed to get container status \"bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243\": rpc error: code = NotFound desc = could not find container \"bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243\": container with ID starting with bda6ea849f64af96110d14b17ac811ca6057f85149d61b2891155a91d6c1b243 not found: ID does not exist" Jul 24 02:19:36.240947 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.240928 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr"] Jul 24 02:19:36.247903 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.247884 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-6bcabd-59f7c6478-5wgfr"] Jul 24 02:19:36.258038 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.257990 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz"] Jul 24 02:19:36.260071 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:36.260052 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-6bcabd-86cc45cd48-fdcdz"] Jul 24 02:19:37.992138 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:37.992093 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f978d28-fd7e-45a6-98dd-d3c575dcd58e" path="/var/lib/kubelet/pods/7f978d28-fd7e-45a6-98dd-d3c575dcd58e/volumes" Jul 24 02:19:37.992504 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:37.992458 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="865b48f8-ce23-4b68-a33c-d8cbb0fdffda" path="/var/lib/kubelet/pods/865b48f8-ce23-4b68-a33c-d8cbb0fdffda/volumes" Jul 24 02:19:52.031009 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:52.030977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:19:52.041091 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:52.041068 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bt6k6_f3172562-d5fd-49f2-b690-e620b7629414/ovn-acl-logging/0.log" Jul 24 02:19:57.736738 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:57.736706 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62"] Jul 24 02:19:57.737232 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:57.736965 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" podUID="c014f80f-1324-4e56-b784-dd80d163e4c9" containerName="payload-processing" containerID="cri-o://3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc" gracePeriod=30 Jul 24 02:19:57.745872 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:57.745843 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw"] Jul 24 02:19:57.746071 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:57.746043 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" podUID="1188575b-2c3d-4bbb-877a-11f4ddd33629" containerName="payload-pre-processing" containerID="cri-o://f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed" gracePeriod=30 Jul 24 02:19:58.015587 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.015556 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:19:58.018840 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.018819 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:19:58.187145 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.187085 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gg6fg\" (UniqueName: \"kubernetes.io/projected/c014f80f-1324-4e56-b784-dd80d163e4c9-kube-api-access-gg6fg\") pod \"c014f80f-1324-4e56-b784-dd80d163e4c9\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " Jul 24 02:19:58.187145 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.187154 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c014f80f-1324-4e56-b784-dd80d163e4c9-plugins-config-volume\") pod \"c014f80f-1324-4e56-b784-dd80d163e4c9\" (UID: \"c014f80f-1324-4e56-b784-dd80d163e4c9\") " Jul 24 02:19:58.187410 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.187268 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1188575b-2c3d-4bbb-877a-11f4ddd33629-plugins-config-volume\") pod \"1188575b-2c3d-4bbb-877a-11f4ddd33629\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " Jul 24 02:19:58.187410 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.187322 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmqqq\" (UniqueName: \"kubernetes.io/projected/1188575b-2c3d-4bbb-877a-11f4ddd33629-kube-api-access-xmqqq\") pod \"1188575b-2c3d-4bbb-877a-11f4ddd33629\" (UID: \"1188575b-2c3d-4bbb-877a-11f4ddd33629\") " Jul 24 02:19:58.187603 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.187575 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c014f80f-1324-4e56-b784-dd80d163e4c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c014f80f-1324-4e56-b784-dd80d163e4c9" (UID: "c014f80f-1324-4e56-b784-dd80d163e4c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:19:58.187677 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.187652 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1188575b-2c3d-4bbb-877a-11f4ddd33629-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1188575b-2c3d-4bbb-877a-11f4ddd33629" (UID: "1188575b-2c3d-4bbb-877a-11f4ddd33629"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 02:19:58.189329 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.189306 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1188575b-2c3d-4bbb-877a-11f4ddd33629-kube-api-access-xmqqq" (OuterVolumeSpecName: "kube-api-access-xmqqq") pod "1188575b-2c3d-4bbb-877a-11f4ddd33629" (UID: "1188575b-2c3d-4bbb-877a-11f4ddd33629"). InnerVolumeSpecName "kube-api-access-xmqqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:19:58.189431 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.189335 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c014f80f-1324-4e56-b784-dd80d163e4c9-kube-api-access-gg6fg" (OuterVolumeSpecName: "kube-api-access-gg6fg") pod "c014f80f-1324-4e56-b784-dd80d163e4c9" (UID: "c014f80f-1324-4e56-b784-dd80d163e4c9"). InnerVolumeSpecName "kube-api-access-gg6fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 02:19:58.288203 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.288085 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gg6fg\" (UniqueName: \"kubernetes.io/projected/c014f80f-1324-4e56-b784-dd80d163e4c9-kube-api-access-gg6fg\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:58.288203 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.288145 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c014f80f-1324-4e56-b784-dd80d163e4c9-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:58.288203 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.288160 2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1188575b-2c3d-4bbb-877a-11f4ddd33629-plugins-config-volume\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:58.288203 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.288173 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmqqq\" (UniqueName: \"kubernetes.io/projected/1188575b-2c3d-4bbb-877a-11f4ddd33629-kube-api-access-xmqqq\") on node \"ip-10-0-138-219.ec2.internal\" DevicePath \"\"" Jul 24 02:19:58.319544 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.319509 2564 generic.go:356] "Generic (PLEG): container finished" podID="1188575b-2c3d-4bbb-877a-11f4ddd33629" containerID="f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed" exitCode=0 Jul 24 02:19:58.319708 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.319553 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" Jul 24 02:19:58.319708 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.319594 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" event={"ID":"1188575b-2c3d-4bbb-877a-11f4ddd33629","Type":"ContainerDied","Data":"f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed"} Jul 24 02:19:58.319708 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.319651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw" event={"ID":"1188575b-2c3d-4bbb-877a-11f4ddd33629","Type":"ContainerDied","Data":"ffa89f6c16250b61b0cf7d948eb4623ad89aaa68b20b5ed9945925364052d383"} Jul 24 02:19:58.319708 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.319687 2564 scope.go:117] "RemoveContainer" containerID="f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed" Jul 24 02:19:58.320906 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.320885 2564 generic.go:356] "Generic (PLEG): container finished" podID="c014f80f-1324-4e56-b784-dd80d163e4c9" containerID="3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc" exitCode=0 Jul 24 02:19:58.321009 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.320921 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" Jul 24 02:19:58.321009 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.320948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" event={"ID":"c014f80f-1324-4e56-b784-dd80d163e4c9","Type":"ContainerDied","Data":"3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc"} Jul 24 02:19:58.321009 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.320988 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62" event={"ID":"c014f80f-1324-4e56-b784-dd80d163e4c9","Type":"ContainerDied","Data":"2cb2caed6e2a98ed92a281a068f59373ad3a898cb3c8b3fdee1db08770d32bf6"} Jul 24 02:19:58.328152 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.328108 2564 scope.go:117] "RemoveContainer" containerID="f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed" Jul 24 02:19:58.328480 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:19:58.328451 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed\": container with ID starting with f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed not found: ID does not exist" containerID="f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed" Jul 24 02:19:58.328591 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.328491 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed"} err="failed to get container status \"f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed\": rpc error: code = NotFound desc = could not find container \"f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed\": container with ID starting with f4a44c9d9c8094ab4e064b2381a12aa87e77b0a1f709e48fc1f83b74999fc1ed not found: ID does not exist" Jul 24 02:19:58.328591 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.328510 2564 scope.go:117] "RemoveContainer" containerID="3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc" Jul 24 02:19:58.336328 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.336312 2564 scope.go:117] "RemoveContainer" containerID="3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc" Jul 24 02:19:58.336569 ip-10-0-138-219 kubenswrapper[2564]: E0724 02:19:58.336549 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc\": container with ID starting with 3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc not found: ID does not exist" containerID="3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc" Jul 24 02:19:58.336617 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.336576 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc"} err="failed to get container status \"3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc\": rpc error: code = NotFound desc = could not find container \"3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc\": container with ID starting with 3f35817670b773b53c55d2b76537638b0a3391e83183f5d51d4d8f557a518ccc not found: ID does not exist" Jul 24 02:19:58.344464 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.344444 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62"] Jul 24 02:19:58.349684 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.349665 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-a7fbc1-7c79d589cd-gjd62"] Jul 24 02:19:58.359994 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.359971 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw"] Jul 24 02:19:58.365528 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:58.365506 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-a7fbc1-76c6dc9fbd-gssbw"] Jul 24 02:19:59.992727 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:59.992691 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1188575b-2c3d-4bbb-877a-11f4ddd33629" path="/var/lib/kubelet/pods/1188575b-2c3d-4bbb-877a-11f4ddd33629/volumes" Jul 24 02:19:59.993097 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:19:59.993042 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c014f80f-1324-4e56-b784-dd80d163e4c9" path="/var/lib/kubelet/pods/c014f80f-1324-4e56-b784-dd80d163e4c9/volumes" Jul 24 02:20:09.516240 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:09.516212 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-758df5668-tdl5s_1f235a10-df42-48f2-a41f-b2e2f095bed7/authorino/0.log" Jul 24 02:20:15.832755 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:15.832724 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59c5678768-4d5c9_8af867d2-6806-4022-ba32-6941d91bfb99/copy-manifests/0.log" Jul 24 02:20:15.841396 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:15.841370 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59c5678768-4d5c9_8af867d2-6806-4022-ba32-6941d91bfb99/manager/0.log" Jul 24 02:20:18.276845 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:18.276813 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-758df5668-tdl5s_1f235a10-df42-48f2-a41f-b2e2f095bed7/authorino/0.log" Jul 24 02:20:18.651924 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:18.651897 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kcw6h_cab51544-1b3f-4af6-9b50-9bf2059aa1af/kuadrant-console-plugin/0.log" Jul 24 02:20:19.862910 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:19.862876 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8544c7785d-bb94r_51487251-cd39-4712-a063-2a7352fdfccb/kube-auth-proxy/0.log" Jul 24 02:20:20.107211 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:20.107182 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xdsj8_c02b265e-5e03-4861-af73-fb3b3f96d129/payload-pre-processing/0.log" Jul 24 02:20:20.827707 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:20.827675 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vx8z4_959f462b-47be-44e3-9047-2191c1771d7e/main/0.log" Jul 24 02:20:21.072199 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:21.072168 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f6knk4_c327dc22-9d25-4072-9dcb-3b04228a12ab/main/0.log" Jul 24 02:20:21.198700 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:21.198673 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lxrdv_aeb6b40e-5703-4cdf-b54e-58c00e5d2645/main/0.log" Jul 24 02:20:21.323608 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:21.323580 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-pdbj4_b2e3536d-82fa-4483-9695-4c3eeeac1ab1/main/0.log" Jul 24 02:20:29.831872 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:29.831837 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zx5s5_f843c8af-66c4-41df-bc00-7f2f53f5d3cd/global-pull-secret-syncer/0.log" Jul 24 02:20:29.923522 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:29.923490 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7g7tz_d1321dd5-5c89-4ca8-bbc2-c7a398ed9940/konnectivity-agent/0.log" Jul 24 02:20:29.995181 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:29.995154 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-219.ec2.internal_1de7be92a0a914801205d3ece54cca0e/haproxy/0.log" Jul 24 02:20:34.537978 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:34.537943 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-758df5668-tdl5s_1f235a10-df42-48f2-a41f-b2e2f095bed7/authorino/0.log" Jul 24 02:20:34.632455 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:34.632421 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kcw6h_cab51544-1b3f-4af6-9b50-9bf2059aa1af/kuadrant-console-plugin/0.log" Jul 24 02:20:37.106169 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:37.106065 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkk89_4a3163a2-a71a-44c0-abc5-8f439bc00a3d/node-exporter/0.log" Jul 24 02:20:37.131046 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:37.131021 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkk89_4a3163a2-a71a-44c0-abc5-8f439bc00a3d/kube-rbac-proxy/0.log" Jul 24 02:20:37.154935 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:37.154912 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jkk89_4a3163a2-a71a-44c0-abc5-8f439bc00a3d/init-textfile/0.log" Jul 24 02:20:40.500413 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500380 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk"] Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500654 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1188575b-2c3d-4bbb-877a-11f4ddd33629" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500664 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1188575b-2c3d-4bbb-877a-11f4ddd33629" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500673 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c014f80f-1324-4e56-b784-dd80d163e4c9" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500679 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c014f80f-1324-4e56-b784-dd80d163e4c9" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500696 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2291e112-d90a-4b4e-82de-2d3ef1cde0ee" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500702 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2291e112-d90a-4b4e-82de-2d3ef1cde0ee" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500709 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="865b48f8-ce23-4b68-a33c-d8cbb0fdffda" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500714 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="865b48f8-ce23-4b68-a33c-d8cbb0fdffda" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500720 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61215352-807a-4a0d-9897-68dd71f200ee" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500725 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="61215352-807a-4a0d-9897-68dd71f200ee" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500736 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ac05d7-abfd-4860-8210-8af316f08358" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500741 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ac05d7-abfd-4860-8210-8af316f08358" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500747 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128e71de-0a63-48b1-a8cf-e91edf2a9bdf" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500752 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="128e71de-0a63-48b1-a8cf-e91edf2a9bdf" containerName="payload-pre-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500758 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f978d28-fd7e-45a6-98dd-d3c575dcd58e" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500763 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f978d28-fd7e-45a6-98dd-d3c575dcd58e" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500769 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" containerName="payload-processing" Jul 24 02:20:40.500787 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500774 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" containerName="payload-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500832 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="128e71de-0a63-48b1-a8cf-e91edf2a9bdf" containerName="payload-pre-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500839 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f978d28-fd7e-45a6-98dd-d3c575dcd58e" containerName="payload-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500845 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="61215352-807a-4a0d-9897-68dd71f200ee" containerName="payload-pre-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500851 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="865b48f8-ce23-4b68-a33c-d8cbb0fdffda" containerName="payload-pre-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500856 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2291e112-d90a-4b4e-82de-2d3ef1cde0ee" containerName="payload-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500863 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1188575b-2c3d-4bbb-877a-11f4ddd33629" containerName="payload-pre-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500868 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ac05d7-abfd-4860-8210-8af316f08358" containerName="payload-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500873 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed0c564-ad6b-4c4b-a3c8-5892c7cb1edc" containerName="payload-processing" Jul 24 02:20:40.501390 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.500877 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c014f80f-1324-4e56-b784-dd80d163e4c9" containerName="payload-processing" Jul 24 02:20:40.503730 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.503709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.506387 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.506367 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m522w\"/\"kube-root-ca.crt\"" Jul 24 02:20:40.507543 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.507522 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m522w\"/\"openshift-service-ca.crt\"" Jul 24 02:20:40.507631 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.507525 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m522w\"/\"default-dockercfg-5qkq6\"" Jul 24 02:20:40.512884 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.512861 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk"] Jul 24 02:20:40.623005 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.622956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-podres\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.623208 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.623023 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv79d\" (UniqueName: \"kubernetes.io/projected/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-kube-api-access-jv79d\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.623208 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.623068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-proc\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.623208 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.623102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-lib-modules\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.623208 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.623151 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-sys\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.723755 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723712 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv79d\" (UniqueName: \"kubernetes.io/projected/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-kube-api-access-jv79d\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.723755 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-proc\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.723966 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-lib-modules\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.723966 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723822 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-sys\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.723966 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-podres\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.723966 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723922 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-proc\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.724158 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-sys\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.724158 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-podres\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.724158 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.723976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-lib-modules\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.731184 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.731163 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv79d\" (UniqueName: \"kubernetes.io/projected/d48cdd11-716d-4261-9a77-0ae26f3cc2ed-kube-api-access-jv79d\") pod \"perf-node-gather-daemonset-tv6pk\" (UID: \"d48cdd11-716d-4261-9a77-0ae26f3cc2ed\") " pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.813793 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.813712 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:40.939458 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:40.939430 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk"] Jul 24 02:20:40.942036 ip-10-0-138-219 kubenswrapper[2564]: W0724 02:20:40.942008 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd48cdd11_716d_4261_9a77_0ae26f3cc2ed.slice/crio-328d844cd87cbf142147ceee376ea89f43a98680a24ccc23d6896b5f781001b8 WatchSource:0}: Error finding container 328d844cd87cbf142147ceee376ea89f43a98680a24ccc23d6896b5f781001b8: Status 404 returned error can't find the container with id 328d844cd87cbf142147ceee376ea89f43a98680a24ccc23d6896b5f781001b8 Jul 24 02:20:41.499108 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:41.499069 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" event={"ID":"d48cdd11-716d-4261-9a77-0ae26f3cc2ed","Type":"ContainerStarted","Data":"367004d4b3aea90faf16bf2e5c9c5fa8c5f6fb8382fdf934fcf79af7f0edebbf"} Jul 24 02:20:41.499482 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:41.499177 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" event={"ID":"d48cdd11-716d-4261-9a77-0ae26f3cc2ed","Type":"ContainerStarted","Data":"328d844cd87cbf142147ceee376ea89f43a98680a24ccc23d6896b5f781001b8"} Jul 24 02:20:41.499482 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:41.499199 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:41.515958 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:41.515910 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" podStartSLOduration=1.515896799 podStartE2EDuration="1.515896799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 02:20:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 02:20:41.51549855 +0000 UTC m=+3049.980632989" watchObservedRunningTime="2026-07-24 02:20:41.515896799 +0000 UTC m=+3049.981031216" Jul 24 02:20:42.174533 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:42.174502 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9mtdc_a2a505e1-5606-461f-814c-b2af6874eaf7/dns/0.log" Jul 24 02:20:42.195996 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:42.195968 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9mtdc_a2a505e1-5606-461f-814c-b2af6874eaf7/kube-rbac-proxy/0.log" Jul 24 02:20:42.311805 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:42.311774 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jh57j_1cc4afbc-9f30-49fb-a31f-2ef91e6f5e8d/dns-node-resolver/0.log" Jul 24 02:20:42.952483 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:42.952446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwnc7_54cc477a-48f2-42bd-a455-5c27655cf4d5/node-ca/0.log" Jul 24 02:20:44.070682 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:44.070602 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8544c7785d-bb94r_51487251-cd39-4712-a063-2a7352fdfccb/kube-auth-proxy/0.log" Jul 24 02:20:44.155603 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:44.155577 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xdsj8_c02b265e-5e03-4861-af73-fb3b3f96d129/payload-pre-processing/0.log" Jul 24 02:20:45.004282 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:45.004255 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hl8p7_85471a1f-0293-4586-8794-1f26399f854a/serve-healthcheck-canary/0.log" Jul 24 02:20:45.661270 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:45.661242 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t59rx_45c169e4-62e2-49f9-bdef-43796a5877f5/kube-rbac-proxy/0.log" Jul 24 02:20:45.683674 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:45.683647 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t59rx_45c169e4-62e2-49f9-bdef-43796a5877f5/exporter/0.log" Jul 24 02:20:45.708004 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:45.707983 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t59rx_45c169e4-62e2-49f9-bdef-43796a5877f5/extractor/0.log" Jul 24 02:20:47.512010 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:47.511974 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m522w/perf-node-gather-daemonset-tv6pk" Jul 24 02:20:48.387324 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:48.387295 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59c5678768-4d5c9_8af867d2-6806-4022-ba32-6941d91bfb99/manager/0.log" Jul 24 02:20:48.411726 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:48.411698 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-59c5678768-4d5c9_8af867d2-6806-4022-ba32-6941d91bfb99/copy-manifests/0.log" Jul 24 02:20:49.940207 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:49.940177 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-745f9c9fb8-7bpjs_0555aa78-1095-45dd-94df-85274404aec8/manager/0.log" Jul 24 02:20:50.004052 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:50.004023 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9rw6f_04acfcca-b610-4b9f-928f-f9b27a6d8ab7/openshift-lws-operator/0.log" Jul 24 02:20:50.822327 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:50.822297 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8544c7785d-bb94r_51487251-cd39-4712-a063-2a7352fdfccb/kube-auth-proxy/0.log" Jul 24 02:20:50.893337 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:50.893313 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xdsj8_c02b265e-5e03-4861-af73-fb3b3f96d129/payload-pre-processing/0.log" Jul 24 02:20:52.466543 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:52.466516 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8544c7785d-bb94r_51487251-cd39-4712-a063-2a7352fdfccb/kube-auth-proxy/0.log" Jul 24 02:20:52.532833 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:52.532805 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-xdsj8_c02b265e-5e03-4861-af73-fb3b3f96d129/payload-pre-processing/0.log" Jul 24 02:20:55.168723 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:55.168692 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-ll8vd_0fe852ed-61d6-4476-8756-a8ddf0a29110/migrator/0.log" Jul 24 02:20:55.189306 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:55.189277 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-ll8vd_0fe852ed-61d6-4476-8756-a8ddf0a29110/graceful-termination/0.log" Jul 24 02:20:56.657983 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.657953 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/kube-multus-additional-cni-plugins/0.log" Jul 24 02:20:56.677717 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.677697 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/egress-router-binary-copy/0.log" Jul 24 02:20:56.697896 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.697876 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/cni-plugins/0.log" Jul 24 02:20:56.719447 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.719422 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/bond-cni-plugin/0.log" Jul 24 02:20:56.740065 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.740043 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/routeoverride-cni/0.log" Jul 24 02:20:56.760345 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.760320 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/whereabouts-cni-bincopy/0.log" Jul 24 02:20:56.779826 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.779803 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fzw96_73544e93-3fd9-4035-bdf1-0e04d8f703d1/whereabouts-cni/0.log" Jul 24 02:20:56.993527 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:56.993455 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jxrgw_ed9ebe55-0c55-4c11-abe4-fd21a66ce8c7/kube-multus/0.log" Jul 24 02:20:57.095734 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:57.095706 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jz7sv_51017506-0417-48e7-9d6d-08a40ca55e44/network-metrics-daemon/0.log" Jul 24 02:20:57.114805 ip-10-0-138-219 kubenswrapper[2564]: I0724 02:20:57.114776 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jz7sv_51017506-0417-48e7-9d6d-08a40ca55e44/kube-rbac-proxy/0.log"